CloudBolt Plug-ins

How Plug-ins are Written

Plug-ins are written in Python 3.6 and stored on the filesystem of the CloudBolt server or externally and loaded dynamically via a URL. They are run on the CloudBolt server and have access to the internal CloudBolt APIs, which read and write data to the CloudBolt database by using Django’s object–relational mapper (ORM). This Python interface is simple to learn, see the section on Resources for Writing Plug-ins for more info.

Structure of a CloudBolt Plug-in

A CloudBolt plug-in modules must contain a function called run(). CloudBolt’s job engine will call that function at particular points during a job’s execution.

run(job, **kwargs)

Is passed a CloudBolt Job object and a set of keyword arguments meant to simplify access to common CloudBolt objects the plug-in might need to interact with.

Helpful keywords arguments are passed to any CloudBolt plug-in run method, but the values will be set to None if the object referenced is not applicable to the context of the hook execution. Keyword arguments include:

  • logger: standard Python Logger object (http://docs.python.org/2/library/logging.html).
  • hook_point: A string representation of the hook_point where the action is called from. If the action is not run in the context of an orchestration, hook_point is None.
  • servers: an iterable collection of all servers that are the target of the plug-in. If there is only one target server, the “server” keyword is also available for simplicity.
  • resources: an iterable collection of all deployed resources that are the target of the plug-in. If there is only one target resource, the “resource” keyword is also available for simplicity.

The run method of CloudBolt plug-ins should always return a tuple of 3 strings: (status, output, errors). On success, the plug-in should return (“”, “”, “”). If a non-empty status string is returned, the job engine will save the status, output, and errors to the job object and terminate any further steps in the job that would have been run. A pre-job plug-in can use this behavior to short circuit the execution of the job if certain prerequisite conditions are not met.

Valid Status Values

These are the valid string values that your plug-in can return for status, along with the corresponding display text that will be shown to the user for the job:

  • ‘SUCCESS’ — ‘Completed successfully’
  • ‘WARNING’ — ‘Completed with warnings’
  • ‘FAILURE’ — ‘Completed with errors’
  • ‘CANCELED’ — ‘Canceled by user’

Note

There are a few types of plug-ins that are exceptions to the structure described here and may have different method names, method signatures, and return values. These include plug-ins in rules, Generated Parameter Options plug-ins, Display Condition Plug-ins for Server Actions, and Discovery Plug-ins on Blueprints.

CloudBolt Plug-in Parameterization

When uploading or setting a URL for a new file, for a CloudBolt plug-in, CloudBolt will scan the file contents for template style variables like {{ variable }} and will automatically create action inputs for each variable it discovers. See Action Input Parameters for more details on auto generated action inputs and Action Context for a complete reference list of objects available to CloudBolt plug-ins on different contexts.

CloudBolt will handle prompting users for those inputs, and then render them in the action before executing it. The inputs will be escaped so they can’t execute arbitrary code while still rendering as expected. However, this depends on the parameters being quoted properly in the action code. Ensure that any string inputs also include surrounding quote marks so the value is properly assigned to its variable in Python. An example: hostname = “{{server.hostname}}”. Otherwise, a user could enter arbitrary Python code into an action input and it would be executed as if it had been written by the author of the plug-in.

New in 8.8

CloudBolt Plug-Ins added support for a new action input type for File Upload in version 8.8. This new action input type currently only works in CloudBolt Plug-Ins used as a Blueprint Action Item. You can take advantage of this input type to allow end-users to select a file they want to see uploaded to an S3 bucket or a server being provisioned as part of the Blueprint deployment. Note that CloudBolt will handle uploading the file to the CloudBolt appliance as part of the Blueprint ordering but is up to the actual code inside the CloudBolt Plug-In to make use of that file for any desired purpose. For example if a Blueprint contains a Server Tier named server_tier and an Action Tier where end-users can select a file from their local client that they want uploaded to the server, the plug-in included in the Blueprint Action Item might look something like:

"""
Sample code to demonstrate the use of a "File Upload Action Input type

It is expected to be added as an action tier to a Bluprint that contains a
'server_tier' that builds a linux server
"""
from django.conf import settings

from common.methods import set_progress


def run(job, blueprint_context, **kwargs):

    server = blueprint_context["server_tier"]["server"]


    file = "{{ file }}"
    if file.startswith(settings.MEDIA_URL):
        set_progress("Converting relative URL to filesystem path")
        file = file.replace(settings.MEDIA_URL, settings.MEDIA_ROOT)

    guest_path = "{{guest_path}}"

    server.copy_file_to_server(file, guest_path)

    return "SUCCESS", "File copy successful!", ""

Action Input Options

Action inputs can be constrained to specific, pre-determined values. This can be done by starting at the action’s details page, clicking on the name of the action input, and then defining global options for that action input.

For action input whose options need to be determined at run time, the options can be provided by the plug-in code. Specifically, for each action input in the plug-in you can define a method called generate_options_for_<action-input-name> that returns the options for that action input.

For example, if we wanted to give the user options for the prefix for the hostname for a server, we could define a variable to use in its run method like prefix = “{{ hostname_prefix }}”. This would create an action input called hostname_prefix. One possible implementation of the method for generating its options is as follows.

def generate_options_for_hostname_prefix(server=None, **kwargs):
  options = [('test', 'test'), ('example', 'example')]
  if server:
     options.append((server.group.id, server.group.name))
     options.append((server.environment.id, server.environment.name))
   return options

The method returns a list of tuples where every tuple contains a value to use and a label to display for the option. The context passed to the method should match the contexts described elsewhere, depending where the plug-in is used. Best practice is to define the method to accept arbitrary kwargs so it can gracefully handle if more items are passed to the context in the future.

It is also possible to define an initial value for the action input as part of the options. Another possible implementation of the method that does so is as follows.

def generate_options_for_hostname_prefix(server=None, **kwargs):
  options = [('test', 'test'), ('example', 'example')]
  if server:
     options.append((server.group.id, server.group.name))
     options.append((server.environment.id, server.environment.name))
   return {'initial_value': server.group.id, 'options': options}

In this case, the method returns a dictionary with keys for the initial value and options, where the options are a list of tuples as before. The initial value should match the first item in one of the option tuples.

These options will appear both when setting default values for the action input and when choosing a value at execution time. If an initial value is provided, it will simply be the option that is first selected when the widget loads.

For more information on all the possible ways to control action input options see the Generated Parameter Options Appendix.

CloudBolt Plug-in Examples

Places to find example CloudBolt Plug-ins: * CloudBolt hosts a content library with a range of example plug-ins. These can be imported in your CloudBolt UI on most pages where actions can be created (ex. the Server Actions page, the Orchestration Actions page). * The code for these plug-ins is stored in a public github repository and can be browsed here: https://github.com/CloudBoltSoftware/cloudbolt-forge/tree/master/actions/cloudbolt_plugins

To demonstrate a few key concepts, a few examples have been included here.

Here’s a simple plug-in, which can be used in Provision Server job’s Pre-Create Resource trigger point . This action simply adds a message to the job progress (and log file) stating the IP address of the server. It then waits for 60 seconds, to give the user watching the job in CloudBolt’s UI a chance to notice this new progress message.

import time
from common.methods import set_progress


def run(job, server=None, **kwargs):
    msg = "In Pre-Create Resource for job #{}, the server's IP is {}".format(job.id, server.ip)
    set_progress(msg)
    time.sleep(60)
    return "","",""

This example plug-in can be used in Provision Server job’s Pre-Create Resource trigger point to change the hostname for the server, prefixing what the user entered with the value of a parameter called org_id.

import sys
from common.methods import set_progress


def run(job, server=None, **kwargs):
    set_progress("In Pre-Create Resource job.id={}".format(job.id))
    # Plug-in to add a prefix to the hostname of any server whose group
    # has an Organization ID parameter associated with it
    agency_id = server.org_id
    if not org_id:
        prog_msg = "Not altering the hostname as no org_id was found."
        set_progress(prog_msg)
        return "","",""
    old_hostname = server.hostname
    server.hostname = "{}{}".format(org_id, server.hostname)
    server.save()
    prog_msg = "Prefixed hostname with Organization ID. {} -> {}".format(
        old_hostname, server.hostname)
    set_progress(prog_msg)

    return "","",""

The next example demonstrates how to use multi-value parameters with a server action that updates a hypothetical “Users” parameter on the server using a “New Users” action input that will be specified when the action is run. Both the parameter and the action input should be set to allow multiple values.

from common.methods import set_progress


def run(job, server=None, **kwargs):
   users = server.users
   new_users = {{ new_users }}
   for user in new_users:
       if user not in users:
           job.set_progress("Adding {}".format(user))
           users.append(user)
   server.users = users
   return "", "", ""

The following is a more advanced example, that should be run at Provision Server job’s Post-Provision trigger point that reads the enable_monitoring parameter and, if it is set to true, emails the CloudBolt admin to let them know. The email step could be changed to email someone else, or even to call into the monitoring system to automatically enable monitoring for the new server. This plug-in should be configured to only run when the job is successful.

"""
For each successful job, if a custom field 'enable_monitoring' is set, send an
email to the CloudBolt admin.
"""
from common.methods import set_progress
from django.conf import settings
from utilities import mail


def run(job, logger, server=None, **kwargs):
    if server.enable_monitoring == True:
        set_progress("Enable monitoring is set for this job.")

        if hasattr(settings, 'ENABLE_MONITORING_EMAIL'):
            subject = render_to_string('email/enable-monitoring-subj.template', {
                'servers': servers}).strip()
            body = render_to_string('email/enable-monitoring-body.template', {
                'owner': job.owner,
                'servers': servers})
            sender = None # use default
            recipient = getattr(settings, 'ENABLE_MONITORING_EMAIL')

            try:
                mail.send_mail(subject, body, sender, [recipient])
            except Exception as err:
                logger.exception("Aborting due to email error")
                return ("FAILURE", "Aborting due to email error. ", str(err))
            #mail utility already logs this, so just update job progress
            set_progress("Email sent to {}".format(recipient))
        else:
            set_progress("Email not sent because customer_settings.ENABLE_MONITORING_EMAIL is not set.")
    else:
        set_progress("Monitoring not enabled.")
    return "", "", ""

Special Case Plug-ins

There are a few special CloudBolt plug-ins, where they might be executed directly from the front-end appliance and outside of the context of a job executions, or they have specific return types that differ from the default CloudBolt Plug-in return tuple. Those include:

Order Approval Plug-ins

When the Order Approval plug-in is enabled, CloudBolt’s default behavior of emailing approvers when an order is submitted is replaced by the logic in the plug-in. This provides a mechanism to integrate CloudBolt with a separate change management system, or to override the standard Order Approval workflow with something that better suits your needs.

CloudBolt’s “Two Approvers” Order Approval Plugin is included by default. Other approval plugins may be found in the Content Library.

Order Submission Actions

When an order is submitted by a user, any enabled and configured “Order Submission” plug-ins are executed. By default, or if no plug-ins are enabled, the outcome of this action will be to set an order to “pending”, requiring that a single user from any group which has approval permission for the requesting group approve the order.

This Orchestration Action allows orders to be automatically approved, automatically denied, or to be assigned to a specific group to approve the order.

Post-Order Approval Actions

When an order is approved by a user, any enabled and configured “Post-Order Approval” plug-ins are executed. By default, or if no plug-ins are enabled, the outcome of this action will be to approve an order, which will then execute any jobs that are spawned.

This Orchestration Action effectively allows one to override the function of the “approve” button. A plug-in could verify that at least two distinct users have approved an order, or it could establish a workflow that requires any order submitted by the “Developers” group to be approved by two users in the “Finance” group, then one user in the “IT” group.

The most useful method for this plug-in is order.set_pending(), which will keep track of the initial approval, but will return the order to a state requiring approval from other users. By calling set_pending(), one also ensures multiple, distinct users will have to approve an order before it becomes active. Note: if a group does not have enough users to approve an order that has had set_pending() called on it, that order may have trouble being approved.

Helpful Methods

There are a number of methods and attributes on the Group and Order classes that may be useful when writing an Order Approval Plugin.

Method Name Function Use Case
Group.get_groups_can_approve() Return all Groups whose orders can be approved by this group, including this Group.  
Group.get_approved_by_groups() Return all Groups who can approve orders submitted by this group, including this Group.  
Order.approve() Approve a pending order. When called through a “Order Submission” action, this method automatically approves an order without input from a user.
Order.deny() Deny a pending order. When called through a “Order Submission” action, this method automatically denies an order without input from a user.
Order.set_pending() Return an active order to pending. When called through a “Post-Order Approval” action, this method will require input (e.g. approval or denial) from another user, while still tracking the approval from the initial user.
Order.groups_for_approval If set to None, returns all groups that have permission to approve this order. If set to a specific group, only the specified group can approve this order.  
Order.approvers Return a list of all unique users who have approved this order. Allows us to verify that a specific user has approved the order, or that a specified number of users have approved the order (e.g. len(order.approvers)).
Order.all_groups_approved(groups) Verifies that all groups specified in the groups argument have approved the order.  

Example Order Submission Plug-in

import sys
import time
from common.methods import set_progress

def run(order, **kwargs):
    set_progress("In Order Submission plug-in order.id={}".format(order.id))
    # randomly approve or deny the order, replace this logic with your own
    if int(time.time()) % 2 == 0:
        order.approve()
        return ("", "Order approved", "")
    else:
        order.deny()
        # note: the plug-in was still successful, so we return an empty status,
        # even though we have rejected the order.
        return ("", "Order denied", "")

Generate Parameter Option Plug-ins

Using a CloudBolt Plug-in to generate options for a parameter is one of the most complex ways to alter the out-of-the-box CloudBolt behavior and warrants it’s own document section. See the Generated Parameter Options Documentation for details.

Auto-Select Environment Plug-ins

Using a CloudBolt Plug-in to determine the best execution venue for a given Server Tier in a Blueprint is one of many powerful features in CloudBolt. Setting up the feature and all the plug-in specific details is covered in detail in the Auto-Select Environment Documentation.

Plug-in Development Tips

Storing Credentials

If you need to use credentials in a plug-in, one way to store them securely is to use Connection Info. Go to Admin > Connection Info and create a Connection Info entry with an easily identifiable name and the information you need. Then in your plug-in you can access and make use of the Connection Info using something like the following code snippet.

connection_info = ConnectionInfo.objects.get(name='my_connection_info')
ip = connection_info.ip
username = connection_info.username
password = connection_info.password

Making requests using the Requests library

Requests is a common library for making HTTP requests in Python. When making requests to HTTPS endpoints, the SSL certificates will be verified against a list of Root Certificates provided by the Certifi library. CloudBolt also provides a way to append additional certificates to the default list, or to disable verification altogether if the CloudBolt installation requires it. Those settings are available at Admin SSL Root Certificates. CloudBolt provides a method for fetching the appropriate settings, which can be used in plug-ins that would work across CloudBolt instances.

import requests
from utilities.helpers import get_ssl_verification
response = requests.get('https://github.com', verify=get_ssl_verification())

Resources for Writing Plug-ins

Interacting with CloudBolt’s Models

  • CloudBolt’s auto-generated model documentation can be seen by navigating to: http://your_CloudBolt_server/alladmin/doc/models/
  • CloudBolt features an interactive shell, which is very useful for testing out commands before adding them to your CloudBolt plug-in. To open this shell:
  1. ssh to the CloudBolt server
  2. Run /opt/cloudbolt/manage.py shell_plus. This will open an interactive Python shell and import all of CloudBolt’s models for you to use
  3. From here, you can interact with CloudBolt’s models. Ex.
# Reboot the server with hostname "devtest0032"
server = Server.objects.get(hostname="devtest0032")
server.reboot()

# Re-assign all of David's servers to Lisa
lisa = UserProfile.objects.get(user__username='lisa')
server.objects.filter(owner__user__username='david').update(owner=lisa)

Danger

Using CloudBolt’s API can be destructive (to CloudBolt and potentially to the servers it manages). We highly recommend doing all plug-in development work in a dev CloudBolt with only development servers under management.

Learning Django’s ORM

CloudBolt’s models are exposed through Django’s Object Relational Model, so understanding this ORM is key to effectively querying CloudBolt.

Learning Python

Advanced: The Plug-in Debugger

The plug-in debugger can be useful for understanding how plug-ins work and why they fail. This can be a richer way to troubleshoot behavior than adding individual logging statements to plug-in code and the information conveyed by the debugger can yield a broader understanding of the program flow.

When a job enters debug mode, the CloudBolt Job Engine will change the state of the job to Paused and the job’s details page in the CB UI will show information about the execution state, including the value of all the variables and the highlighted line of code where the execution is currently paused. CB Admins can then choose to continue execution in several ways (ex. until the next line, until the end of the current method), resume normal execution, or cancel the job.

How to Enable Debug Mode

Debug mode can be enabled in two different ways:

  1. (easier) Log in to the CB UI as an admin user, navigate to the details page for your plug-in. There is a toggle switch in the upper right part of the page that you can use to change whether the debugger is enabled or not. Debugging will start before the first line of the plug-in runs.
  2. (allows starting debug mode at the beginning of any method) Edit your plug-in’s code to import CloudBolt’s settrace decorator, and then add @settrace on the line above the method that you want to debug. To debug from the very beginning of execution,
# Put this line at the top of your plug-in, with your other imports
from jobs.tracer import settrace

@settrace
def method_you_want_to_trace():
    ...

Tracing methods that generate options for parameters is not currently supported, since those do not run within the context of a job.

Next Steps

This section covers the basics of CloudBolt plug-in authoring, but if you have any additional questions or want to discuss a more specific use case you have, we would love to hear from you. Just send us a note at support@cloudboltsoftware.com with your preferred method of contact.