Catalog

Overview

The Catalog enables admins to create blueprints for controlled and repeatable deployments by end users.

A blueprint can be as simple as a one-click single server build and as complex as a multi-tier load balanced application stack running in multiple environments.

A blueprint may even be composed of only actions, with no servers provisioned at all (ex. creating a new LUN by running a CB plugin that interacts with a storage API). Admins can use this flexibility to implement “X as a Service” or XaaS. For example, one could create blueprints for ordering laptops, filing support tickets in an external tracker, or similar solutions not involving actual VM provisioning. Blueprint managers can have their blueprint actually create a resource object to represent the result of ordering that blueprint, such as a laptop, or make no overall resource (e.g., for filing a support ticket, where there’s no need to keep a record of that object in CB). The types of resources available are defined by CloudBolt Admins on the Admin > Resource Types (XaaS) page.

../_images/catalog.png

Blueprints & Resource Lifecycle

A blueprint defines the build process, management actions, and teardown process of a resource. Blueprints can also be configured to not create a resource when ordered, in which case they only define which servers to build and/or actions to run.

../_images/multitier-blueprint.png

When a new resource is ordered, build blueprint items may create server tiers, run remote scripts on servers, deploy a pod of containers, enact networking changes (such as creating a virtual network or load balancer), email users, connect with external systems, run CloudBolt plugins, and deploy sub-blueprints that create any type of resource (or none).

Once a resource is deployed, ongoing management may involve resource actions that users can run. These are defined either on the blueprint Management tab (specific to this blueprint) or in Admin > Resource Actions (general purpose).

Finally, when a resource is deleted, actions may be defined on the blueprint Teardown tab to clean up external resources, remove DNS entries, and so forth.

Managers can duplicate a blueprint to easily create similar versions of it, as well as export and import blueprints for version control or sharing between dev and production CloudBolt environments.

Server Tiers

Server tiers are blueprint items that deploy one or more servers of a particular role. Usually, the OS build and some other primary attributes such as hostname template are predetermined by the admin. Other attributes can be chosen by end users at order time, within the options and constraints specified by the target environment.

These items can be enabled for a single environment or multiple environments, with different parameter options available for each environment.

The following illustrate the flexibility offered by blueprints with regard to how much choice end users have at install time:

  • A single Windows server for a specific environment, always with two CPUs 4 GB of memory and on the same dev VLAN.
  • A single Ubuntu server to the user’s choice of 10 different environments, where the hardware resources change depending on environment. E.g. the server would have one CPU and 2 GB of memory in a VMware environment and an instance type of t1.micro in an AWS environment.
  • A tier that leaves all of these choices up to the end-user: the OS, environment, the quantity of servers to deploy, the environment, resources, network, etc. This is the type of blueprint used by the New Server link.

Cost Preview

For blueprints with a single Server Tier that has specific environments configured (as opposed to when admins simply allow all capable environments) the order form will show end users a cost comparison chart for each environment. This preview is based on the base cost of each environment choice.

Action Blueprint Items

Actions can be added as items within a blueprint to perform any type of operation that is automatable. These actions can be one of any of the five types supported elsewhere in the product and can run code on servers in specific tiers that have just been deployed (remote script actions), on the CB server itself (CloudBolt Plug-ins), and integrate with external systems.

Specific examples of what action blueprint items can do:

  • Email the monitoring team to let them know there is a new deployed resource that should be monitored
  • Connect to the backup system to automatically enable backups for servers deployed by this blueprint
  • Create a new Azure website (see sample scripts in the CloudBolt Forge)
  • Send a message to your team’s chat room (see sample WebHooks in the CB Forge)

These action blueprint items can provide everything they need at runtime or they can use values from servers in other tiers (ex. the IP of the DB server that was deployed earlier in the blueprint deployment). Actions also can ask the user for inputs at runtime. See Action Input Parameters for more information on action inputs, and Action Context for details on what context is provided to actions.

Note

Only CloudBolt Admins can create or edit CloudBolt Plug-ins, even in the context of a Blueprint. Blueprint managers have more control over the other action types. However, if they are not a CB Admin they will not be able to share any actions or edit actions that are shared.

Blueprint Sub-component Items

New in 7.0

Admins can use the Catalog to define Blueprints that can be included as items in other blueprints. Those sub-component blueprints can be as simple or as complex as needed, as long as they meet the following criteria:

  • The sub-blueprints are configured so ANY group can deploy them.
  • The sub-blueprints don’t contain any sub-blueprints.

Note that because there can only be one level of sub-blueprints, a blueprint that is already used as a sub-blueprint will not present the option to add sub-blueprints on its build tab.

If an admin would like to make a particular catalog blueprint available only as a sub-component, it is possible to do so by setting the ‘Is Orderable’ setting on the Blueprint to false.

Blueprint-level Parameters

Blueprint managers have the ability to associate parameters with blueprints using the Parameters tab when managing the blueprint. These parameters will appear when ordering the blueprint, and the values entered will be associated with each build item in the deployed resource, the deployed resource itself, or both, depending on the destination that the manager selects. For example, this could be used to easily set the same Expiration Date for all servers provisioned by the blueprint or to mark the entire resource as being associated with a particular ticket number.

On the Parameters tab when managing the blueprint, it is possible to set options and constraints for a parameter at the blueprint level, just like you can do for Environments and Groups. These options and constraints, however, only apply to the particular blueprint and take precedence over any options or constraints set somewhere else.

If a blueprint-level parameter includes build items as a destination, it will not appear on the form when configuring or ordering individual build items in that blueprint. Note that a preconfiguration will still show on a server blueprint item even if it includes a parameter set at the blueprint level. This is because the preconfiguration likely includes other parameters as well. The blueprint-level value will override the preconfiguration value for that one parameter. For example, if 3 is entered for a blueprint-level Mem Size but small is selected for a Server Size preconfiguration (which translates to 1 CPU and 1 GB memory), the resulting server(s) will have 1 CPU and 3 GB memory.

Resource Names

If desired, Blueprint managers can configure a resource name template that will be used to determine the name of the deployed resource when a blueprint is ordered. Without this setting, the default is to simply use the blueprint name with a unique number. To assign a resource name template to a blueprint, click the Edit button when managing the blueprint. The context available to this template is the blueprint itself with all its attributes, blueprint-level parameters, and the group selected for the order. It uses the Django template format and the same uniquification scheme as elsewhere in CB, such as parameter templates.

For example, to name the resource after the group and the value entered for a blueprint-level parameter named ticket_number, enter {{ group.name }}-{{ blueprint.ticket_number }}-00X. When the blueprint is ordered, assuming it’s configured to create a resource, the deployed resource will end up with a name like “Acme-42-003”.

Ordering

End users order blueprints through the Catalog view. They will see all blueprints that they have permission to order.

Catalog Organization

Your catalog may contain dozens or even hundreds of blueprints. To help users find and discover blueprints, there is a toolbar of filters that makes it easy to search by resource types, labels, or OS build. Users can also search for any blueprint by name or description. The catalog can be sorted alphabetically, by sequence (a weight defined by blueprint managers), by popularity, or how recently they have been added.

../_images/catalog-filters.png

Labels are defined by Super Admins and CloudBolt Admins at Admin > Catalog Management, and then assigned to blueprints by admins or blueprint managers on the Overview tab of a blueprint.

Order Form

Once the blueprint has been set up, end users can order it simply by navigating to the catalog in the CloudBolt user interface and clicking on that blueprint.

If the blueprint has all choices predefined, the user will just need to click the “Submit” button. In fact, blueprint items can be configured to only show if needed, and then the user won’t even have to see them at order time if all the choices are predefined. Otherwise, they may be asked to specify group, environment, hardware and software choices, etc.

If rates are enabled, the order form shows a cost preview comparison chart so users can make an educated decision about which environment to deploy to. This is currently supported for blueprints with a single Server Tier that has been configured for specific environments.

CloudBolt Admins can customize the order in which parameters are shown in blueprint order forms. For example, username fields should typically be grouped alongside and before password fields; hardware resource fields should be grouped together; Quantity should come before most other parameters, and so forth. This is managed in Admin > Orders > Parameter Display Sequence.

When the “Submit” button is clicked, a new order will be created and submitted for approval. If you already have a current order with unsubmitted items, it will be unaffected.

If the order is not auto-approved, an approver can edit it before approving. Similarly, if an order is later duplicated, the owner can edit the new order before submitting it. Note that the current configuration of any associated blueprints, environments, etc. will take precedence over the details of the order being edited if they are different. Also note that if a pre-7.4 order is duplicated, the resulting order may not be editable.

Blueprints that do not Create Resources

Blueprints can also be ordered without creating a deployed resource, if and only if they consist entirely of server build items and/or action build items. If the “Resource Type” in the Edit dialog is None, when a user orders the blueprint it will simply provision the defined server build items and/or run the defined action build items, without doing some resource-related steps. Many of the concepts mentioned elsewhere in this document, such as blueprint-level parameters, resource names, and the resource lifecycle, will not apply when the blueprint does not create a resource. This is the type of blueprint used by the New Server link.

Informational Blueprints

Some IT shops use CloudBolt’s Catalog as the primary place users go to request everything. Some of these blueprints are traditional resources that CB builds, but others are things that can only be ordered from other systems. For these, CB admins can set up informational blueprints, that users can find in the catalog, but the description of the blueprint (which can contain HTML) links to another system. To set one of these up, simply create a blueprint with no build items, but a good description and image, and share it with whichever groups should see it. When “ordering” it, blueprint admins will see a message stating that the blueprint has no build items.

Exporting and Importing Blueprints

Blueprints can be exported as files in order to be easily reused across CloudBolt instances or shared in the CloudBolt Content Library, CloudBolt Forge, a repository of sample CloudBolt content. Exported blueprints are typically stored as zip files containing JSON-formatted metadata about the blueprint along with related items and actions.

It’s possible to export both from the details page of a blueprint in the UI and using the API. When exporting, you can choose between a sanitized or instance-specific version of the blueprint, where the sanitized version will omit potentially sensitive information or details that may not make sense when importing into a different CloudBolt, such as environments and groups. Blueprints can be imported from the Catalog page or using the API, and have an option to either replace the existing blueprint or create a new one.

Exports contain all build and teardown blueprint items, all blueprint-specific resource actions, and all action scripts used by these. Note that unless you choose to replace the existing blueprint, new versions of all actions will be created on import. If you’re importing with the replace existing option, it will look for actions that match the attributes appropriate to that specific type of action and use those if they exist.

Exports will also contain all the necessary details on any blueprint-level parameters and, if exported with instance-specific info, enabled environments for blueprint items along with any predefined parameters. The import process for parameters will look for existing matching parameters to use, and create new ones if none are found. However, import of enabled environments will only happen if an environment with the same name is found. This is due to the more complex, instance-specific, and less user-driven nature of environments. Groups will behave the same way as environments.

Similarly, exports of blueprints that create a Resource will contain all the necessary information on the selected Resource Type. The import process will look for an existing matching Resource Type to use, and create a new one if none is found.

Check out the CloudBolt Forge for some example blueprints.

Note

Because only CloudBolt Admins can create or edit CloudBolt Plug-ins, if a Blueprint manager who is not in one of these global roles tries to import a Blueprint that contains one or more CloudBolt Plug-ins, they will receive a permissions error and not be able to do so. Similarly, because non-CB Admins cannot create or edit shared actions, they will not be able to import a Blueprint that contains a shared action or would replace an existing shared action (because they’ve chosen to replace existing), regardless of the action type. Such users can still import Blueprints that do not contain any CloudBolt Plug-ins and would not create or replace shared actions, and CB Admins can import all Blueprints as long as they also have Blueprint management permissions.

Deployed Resources

Once a resource has been installed, it can be managed in the CloudBolt user interface, including scaling it up or down, executing arbitrary actions on it, and eventually deleting it.

Resource Actions

Resource Actions are a mechanism for adding arbitrary actions to deployed resources, via buttons on their details view. For any action that end-users may need to invoke on deployed resources, CloudBolt admins can add a resource action to appear on the deployed resource’s detail page and, when clicked, run one of the five types of CloudBolt actions.

These actions can run scripts on all or a subset of server tiers in the resource, contact external systems, email users, and run code that interacts with the CB database to read and store information on the resource. For examples of resource actions, check the CloudBolt forge or ask your CloudBolt System Engineer.

Actions can be restricted to particular blueprints, so that they will only be shown on resources deployed from those blueprints. For example, a “Restart Web Server” action could be created and associated with the web server blueprint, and that button would only appear on resources deployed from that blueprint.

Creating Attributes on Resources

CloudBolt administrators have the ability to add arbitrary name/value pairs to the details page for deployed resources. For example, when deploying a website resource admins may want to store and display the port that website is listening on, or any other data the user may need to interact with the site.

This is achieved through a plug-in action build item on a blueprint that runs during deployment. These plug-ins can associate arbitrary name value pairs with deployed resources and they will be seen by users viewing the overview tab of the resource. There are examples of associating this metadata with deployed resources in the CloudBolt forge.

Auto Scaling

Deployed resources can be configured so that their tiers scale based on CPU load (and other conditions).

Overview

CloudBolt can run actions to detect CPU load on a tier of servers in a deployed resource, and if a maximum threshold is reached, it can initiate the building of new servers to add to that tier and then update the load balancer to add the new servers to the rotation. Later, when CloudBolt detects that the load is below a minimum threshold, it can delete servers in the tier, automatically removing them from the load balancer rotation as it does so.

The auto scaling feature has been built using CloudBolt’s rules engine, with its central logic contained in CloudBolt Plugins so that the behavior is transparent and extensible. CloudBolt customers can override this logic if they would like auto scaling to behave differently than it does out of the box.

Prerequisites

Before beginning with auto scaling, build a simple blueprint that consists of one or more servers and optionally a load balancer. A blueprint that deploys two simple web servers with some static content and a load balancer directing traffic to the two servers would be ideal. Then deploy this blueprint, ensure the resultant resource is behaving as expected, and that running manual scale up and scale down operations from the CloudBolt UI works properly.

How to Configure Auto Scaling

  1. To enable the auto-scaling rule and add the actions and parameters necessary for this, ssh to the CloudBolt server as root and run /opt/cloudbolt/initialize/create_objects.py /opt/cloudbolt/initialize/cloud_bursting.py
  2. In the CB UI, navigate to Admin > Actions and filter to actions that contain “Get CPU”.
  3. ^Click on “Get CPU Utilization for VMware”, edit it, and restrict it to the VMware resource technology
  4. ^Go back to the list of actions, click on “Get CPU Utilization for AWS”, edit it, and restrict it to the AWS resource technology
  5. In the CB UI, navigate to the resource you would like to auto scale, click the parameters tab, and add the parameter “Auto-Scaling Config”. Here is a sample auto-scaling configuration value to use (replacing the capitalized sections):
{
    "TIER NAME": {
        "ENVIRONMENT NAME": {
            "cpu_max_threshold": 40,
            "cpu_min_threshold": 10,
            "max_servers": 5,
            "min_servers": 1,
            "scale_by": 1
        }
    }
}

^-this step will not be necessary in future versions of CloudBolt

How to Test

First, test scaling up:

  1. Add the “Simulated CPU Load” parameter to the resource, giving it a value of 50
  2. Open a separate tab in your browser and navigate to Admin > Rules
  3. Run the “Check Resources for Auto-Scaling Conditions” Rule
  4. Click on the job that was created. It should detect that the load is beyond the threshold and initiate a job to provision another web server
  5. While the provisioning job is running, return to your browser tab with the resource details. Click on the overview tab for the resource and note the depiction of a new server being built for the resource
  6. After the provisioning is done, you can confirm that the load balancer has been updated to include the new server

Then test scaling back down:

  1. Change the “Simulated CPU Load” parameter’s value to 5
  2. Run the “Check Resources for Auto-Scaling Conditions” Rule again
  3. Click on the new job. This check should initiate a scale down job to delete one of the servers in the resource

Then, to clean up, remove the “Simulated CPU Load” parameter from the resource.

Lastly, test load detection with real CPU load:

  1. Generate significant CPU load on the servers in the scaling tier
  2. Run the rule
  3. Ensure that the rule scales the tier up
  4. Reduce the load
  5. Run the rule
  6. Ensure that the rule scales the tier back down

Bursting Configuration

In order to cause a tier to scale up into a different environment:

  1. In the blueprint, configure that server blueprint item to be deployed into another environment
  2. In a deployed resource, in the Auto-Scaling Config, specify a second environment for the tier
  3. Add a “burst_into” key in the first environment to cause it to scale into the second environment after the first is at its max
{
    "TIER NAME": {
        "ENVIRONMENT 1 NAME": {
            "cpu_max_threshold": 20,
            "cpu_min_threshold": 10,
            "max_servers": 2,
            "min_servers": 1,
            "scale_by": 1,
            "burst_into": "ENVIRONMENT 2 NAME"
        },
        "ENVIRONMENT 2 NAME": {
            "cpu_max_threshold": 50,
            "cpu_min_threshold": 20,
            "max_servers": 3,
            "min_servers": 0,
            "scale_by": 1
        }
    }
}

After the second environment has been burst into, it will be scaled up and down according to its own thresholds (and will also grow if the first environment bursts again).