Catalog

Overview

The Catalog enables admins to create blueprints for controlled and repeatable deployments by end users.

A blueprint can be as simple as a one-click single server build and as complex as a multi-tier load balanced application stack running in multiple environments.

A blueprint may even be composed of only actions, with no servers provisioned at all (ex. creating a new LUN by running a CB plugin that interacts with a storage API). Admins can use this flexibility to implement “X as a service” or XaaS. For example, one could create blueprints for ordering laptops, filing support tickets in an external tracker, or similar solutions not involving actual VM provisioning.

../_images/catalog.png

Blueprints & Service Lifecycle

A blueprint defines the build process, management actions, and teardown process of a service. Blueprints can also be configured to not create a service when ordered, in which case they only define which servers to build and/or actions to run.

../_images/multitier-blueprint.png

When a new service is ordered, build blueprint items may create server tiers, run remote scripts on servers, deploy a pod of containers, enact networking changes (such as creating a virtual network or load balancer), email users, connect with external systems, and run CloudBolt plugins.

Once a service is deployed, ongoing management may involve service actions that users can run. These are defined either on the blueprint Management tab (specific to this blueprint) or in Admin > Service Actions (general purpose).

Finally, when a service is deleted, actions may be defined on the blueprint Teardown tab to clean up external resources, remove DNS entries, and so forth.

Managers can duplicate a blueprint to easily create similar versions of it, as well as export and import blueprints for version control or sharing between dev and production CloudBolt environments.

Server Tiers

Server tiers are blueprint items that deploy one or more servers of a particular role. Usually, the OS build and some other primary attributes such as hostname template are predetermined by the admin. Other attributes can be chosen by end users at order time, within the options and constraints specified by the target environment.

These items can be enabled for a single environment or multiple environments, with different parameter options available for each environment.

The following illustrate the flexibility offered by blueprints with regard to how much choice end users have at install time:

  • A single Windows server for a specific environment, always with two CPUs 4 GB of memory and on the same dev VLAN.
  • A single Ubuntu server to the user’s choice of 10 different environments, where the hardware resources change depending on environment. E.g. the server would have one CPU and 2 GB of memory in a VMware environment and an instance type of t1.micro in an AWS environment.
  • A tier that leaves all of these choices up to the end-user: the OS, environment, the quantity of servers to deploy, the environment, resources, network, etc. This is the type of blueprint used by the New Server link.

Cost Preview

For blueprints with a single Server Tier that has specific environments configured (as opposed to when admins simply allow all capable environments) the order form will show end users a cost comparison chart for each environment. This preview is based on the base cost of each environment choice.

Action Blueprint Items

Actions can be added as items within a blueprint to perform any type of operation that is automatable. These actions can be one of any of the five types supported elsewhere in the product and can run code on servers in specific tiers that have just been deployed (remote script actions), on the CB server itself (CloudBolt Plug-ins), and integrate with external systems.

Specific examples of what action blueprint items can do:

  • Email the monitoring team to let them know there is a new deployed service that should be monitored
  • Connect to the backup system to automatically enable backups for servers deployed by this blueprint
  • Create a new Azure website (see sample scripts in the CloudBolt Forge)
  • Send a message to your team’s chat room (see sample WebHooks in the CB Forge)

These action blueprint items can provide everything they need at runtime or they can use values from servers in other tiers (ex. the IP of the DB server that was deployed earlier in the service installation). Actions also can ask the user for inputs at runtime. See Action Input Parameters for more information on action inputs, and Action Context for details on what context is provided to actions.

Note

Only CloudBolt Admins can create or edit CloudBolt Plug-ins, even in the context of a Blueprint. Blueprint managers have more control over the other action types. However, if they are not a CB Admin they will not be able to share any actions or edit actions that are shared.

Blueprint Sub-component Items

New in 7.0

Admins can use the Catalog to define Blueprints that can be included as items in other blueprints. Those sub-component blueprints can be as simple or as complex as needed, as long as they meet the following criteria:

  • The sub-blueprints are configured so ANY group can deploy them.
  • The sub-blueprints don’t contain any sub-blueprints.

If an admin would like to make a particular catalog blueprint available only as a sub-component, it is possible to do so by setting the ‘Is Orderable’ setting on the Blueprint to false.

Blueprint-level Parameters

Blueprint managers have the ability to associate parameters with blueprints using the Parameters tab when managing the blueprint. These parameters will appear when ordering the blueprint, and the values entered will be associated with each service item in the deployed service, the deployed service itself, or both, depending on the destination that the manager selects. For example, this could be used to easily set the same Expiration Date for all servers provisioned by the blueprint or to mark the entire service as being associated with a particular ticket number.

On the Parameters tab when managing the blueprint, it is possible to set options and constraints for a parameter at the blueprint level, just like you can do for Environments and Groups. These options and constraints, however, only apply to the particular blueprint and take precedence over any options or constraints set somewhere else.

If a parameter is associated at the blueprint level, it will not appear on the order form for a particular server blueprint item. Note that a preconfiguration will still show on a server blueprint item even if it includes a parameter set at the blueprint level. This is because the preconfiguration likely includes other parameters as well. The blueprint-level value will override the preconfiguration value for that one parameter. For example, if 3 is entered for a blueprint-level Mem Size but small is selected for a Server Size preconfiguration (which translates to 1 CPU and 1 GB memory), the resulting server(s) will have 1 CPU and 3 GB memory.

Service Names

If desired, Blueprint managers can configure a service name template that will be used to determine the name of the deployed service when a blueprint is ordered. Without this setting, the default is to simply use the blueprint name with a unique number. To assign a service name template to a blueprint, click the Edit button when managing the blueprint. The context available to this template is the blueprint itself with all its attributes, blueprint-level parameters, and the group selected for the order. It uses the Django template format and the same uniquification scheme as elsewhere in CB, such as parameter templates.

For example, to name the service after the group and the value entered for a blueprint-level parameter named ticket_number, enter {{ group.name }}-{{ ticket_number }}-00X. When the blueprint is ordered, assuming it’s configured to create a service, the deployed service will end up with a name like “Acme-42-003”.

Ordering

Once the blueprint has been set up, end users can order it simply by navigating to the catalog in the CloudBolt user interface and clicking on that blueprint.

If the blueprint has all choices predefined, the user will just need to click the “Submit” button. In fact, blueprint items can be configured to only show if needed, and then the user won’t even have to see them at order time if all the choices are predefined. Otherwise, they may be asked to specify group, environment, hardware and software choices, etc.

If rates are enabled, the order form shows a cost preview comparison chart so users can make an educated decision about which environment to deploy to. This is currently supported for blueprints with a single Server Tier that has been configured for specific environments.

CloudBolt Admins can customize the order in which parameters are shown in blueprint order forms. For example, username fields should typically be grouped alongside and before password fields; hardware resource fields should be grouped together; Quantity should come before most other parameters, and so forth. This is managed in Admin > Orders > Parameter Display Sequence.

When the “Submit” button is clicked, a new order will be created and submitted for approval. If you already have a current order with unsubmitted items, it will be unaffected.

If the order is not auto-approved, an approver can edit it before approving. Similarly, if an order is later duplicated, the owner can edit the new order before submitting it. Note that the current configuration of any associated blueprints, environments, etc. will take precedence over the details of the order being edited if they are different. Also note that if a pre-7.4 order is duplicated, the resulting order may not be editable.

Blueprints that do not Create Services

Blueprints can also be ordered without creating a deployed service, if and only if they consist entirely of server build items and/or action build items. If the “Create service” option in the Edit dialog is unchecked, when a user orders the blueprint it will simply provision the defined server build items and/or run the defined action build items, without doing some service-related steps. Many of the concepts mentioned elsewhere in this document, such as blueprint-level parameters, service names, and the service lifecycle, will not apply when this option is chosen. This is the type of blueprint used by the New Server link.

Informational Blueprints

Some IT shops use CloudBolt’s Catalog as the primary place users go to request everything. Some of these blueprints are traditional services that CB builds, but others are things that can only be ordered from other systems. For these, CB admins can set up informational blueprints, that users can find in the catalog, but the description of the blueprint (which can contain HTML) links to another system. To set one of these up, simply create a blueprint with no build items, but a good description and image, and share it with whichever groups should see it. When “ordering” it, blueprint admins will see a message stating that the blueprint has no build items.

Exporting and Importing Blueprints

Blueprints can be exported as files in order to be easily reused across CloudBolt instances or shared in the CloudBolt Content Library, CloudBolt Forge, a repository of sample CloudBolt content. Exported blueprints are typically stored as zip files containing JSON-formatted metadata about the blueprint along with related items and actions.

It’s possible to export both from the details page of a blueprint in the UI and using the API. When exporting, you can choose between a sanitized or instance-specific version of the blueprint, where the sanitized version will omit potentially sensitive information or details that may not make sense when importing into a different CloudBolt, such as environments and groups. Blueprints can be imported from the Catalog page or using the API, and have an option to either replace the existing blueprint or create a new one.

Exports contain all build and teardown blueprint items, all blueprint-specific service actions, and all action scripts used by these. Note that unless you choose to replace the existing blueprint, new versions of all actions will be created on import. If you’re importing with the replace existing option, it will look for actions that match the attributes appropriate to that specific type of action and use those if they exist.

Exports will also contain all the necessary details on any blueprint-level parameters and, if exported with instance-specific info, enabled environments for blueprint items along with any predefined parameters. The import process for parameters will look for existing matching parameters to use, and create new ones if none are found. However, import of enabled environments will only happen if an environment with the same name is found. This is due to the more complex, instance-specific, and less user-driven nature of environments. Groups will behave the same way as environments.

Check out the CloudBolt Forge for some example blueprints.

Note

Because only CloudBolt Admins can create or edit CloudBolt Plug-ins, if a Blueprint manager who is not a CB Admin tries to import a Blueprint that contains one or more CloudBolt Plug-ins, they will receive a permissions error and not be able to do so. Similarly, because non-CB Admins cannot create or edit shared actions, they will not be able to import a Blueprint that contains a shared action or would replace an existing shared action (because they’ve chosen to replace existing), regardless of the action type. Such users can still import Blueprints that do not contain any CloudBolt Plug-ins and would not create or replace shared actions, and CB Admins can import all Blueprints as long as they also have Blueprint management permissions.

Deployed Services

Once a service has been installed, it can be managed in the CloudBolt user interface, including scaling it up or down, executing arbitrary actions on it, and eventually deleting it.

Service Actions

Service Actions are a mechanism for adding arbitrary actions to deployed services, via buttons on their details view. For any action that end-users may need to invoke on deployed services, CloudBolt admins can add a service action to appear on the deployed service’s detail page and, when clicked, run one of the five types of CloudBolt actions.

These actions can run scripts on all or a subset of server tiers in the service, contact external systems, email users, and run code that interacts with the CB database to read and store information on the service. For examples of service actions, check the CloudBolt forge or ask your CloudBolt System Engineer.

Actions can be restricted to particular blueprints, so that they will only be shown on services deployed from those blueprints. For example, a “Restart Web Server” action could be created and associated with the web server blueprint, and that button would only appear on services deployed from that blueprint.

Creating Attributes on Services

CloudBolt administrators have the ability to add arbitrary name/value pairs to the details page for deployed services. For example, when deploying a website service admins may want to store and display the port that website is listening on, or any other data the user may need to interact with the site.

This is achieved through a plug-in action blueprint item on a blueprint that runs during service installation. These plug-ins can associate arbitrary name value pairs with deployed services and they will be seen by users viewing the overview tab of the service. There are examples of associating this metadata with deployed services in the CloudBolt forge.

Auto Scaling

Deployed services can be configured so that their tiers scale based on CPU load (and other conditions).

Overview

CloudBolt can run actions to detect CPU load on a tier of servers in a deployed service, and if a maximum threshold is reached, it can initiate the building of new servers to add to that tier and then update the load balancer to add the new servers to the rotation. Later, when CloudBolt detects that the load is below a minimum threshold, it can delete servers in the tier, automatically removing them from the load balancer rotation as it does so.

The auto scaling feature has been built using CloudBolt’s rules engine, with its central logic contained in CloudBolt Plugins so that the behavior is transparent and extensible. CloudBolt customers can override this logic if they would like auto scaling to behave differently than it does out of the box.

Prerequisites

Before beginning with auto scaling, build a simple blueprint that consists of one or more servers and optionally a load balancer. A blueprint that deploys two simple web servers with some static content and a load balancer directing traffic to the two servers would be ideal. Then deploy this blueprint, ensure the resultant service is behaving as expected, and that running manual scale up and scale down operations from the CloudBolt UI works properly.

How to Configure Auto Scaling

  1. To enable the auto-scaling rule and add the actions and parameters necessary for this, ssh to the CloudBolt server as root and run /opt/cloudbolt/initialize/create_objects.py /opt/cloudbolt/initialize/cloud_bursting.py
  2. In the CB UI, navigate to Admin > Actions and filter to actions that contain “Get CPU”.
  3. ^Click on “Get CPU Utilization for VMware”, edit it, and restrict it to the VMware resource technology
  4. ^Go back to the list of actions, click on “Get CPU Utilization for AWS”, edit it, and restrict it to the AWS resource technology
  5. In the CB UI, navigate to the service you would like to auto scale, click the parameters tab, and add the parameter “Auto-Scaling Config”. Here is a sample auto-scaling configuration value to use (replacing the capitalized sections):
{
    "TIER NAME": {
        "ENVIRONMENT NAME": {
            "cpu_max_threshold": 40,
            "cpu_min_threshold": 10,
            "max_servers": 5,
            "min_servers": 1,
            "scale_by": 1
        }
    }
}

^-this step will not be necessary in future versions of CloudBolt

How to Test

First, test scaling up:

  1. Add another parameter to the service called “Simulated CPU Load”, giving it a value of 50
  2. Open a separate tab in your browser and navigate to Admin > Rules
  3. Run the “Check Services for Auto-Scaling Conditions” Rule
  4. Click on the job that was created. It should detect that the load is beyond the threshold and initiate a job to provision another web server
  5. While the provisioning job is running, return to your browser tab with the service details. Click on the overview tab for the service and note the depiction of a new server being built for the service
  6. After the provisioning is done, you can confirm that the load balancer has been updated to include the new server

Then test scaling back down:

  1. Change the simulated load parameter’s value to 5
  2. Run the “Check Services for Auto-Scaling Conditions” Rule again
  3. Click on the new job. This check should initiate a scale down job to delete one of the servers in the service

Then, to clean up, remove the simulated load parameter from the service.

Lastly, test load detection with real CPU load:

  1. Generate significant CPU load on the servers in the scaling tier
  2. Run the rule
  3. Ensure that the rule scales the tier up
  4. Reduce the load
  5. Run the rule
  6. Ensure that the rule scales the tier back down

Bursting Configuration

In order to cause a tier to scale up into a different environment:

  1. In the blueprint, configure that server blueprint item to be deployed into another environment
  2. In a deployed service, in the Auto-Scaling Config, specify a second environment for the tier
  3. Add a “burst_into” key in the first environment to cause it to scale into the second environment after the first is at its max
{
    "TIER NAME": {
        "ENVIRONMENT 1 NAME": {
            "cpu_max_threshold": 20,
            "cpu_min_threshold": 10,
            "max_servers": 2,
            "min_servers": 1,
            "scale_by": 1,
            "burst_into": "ENVIRONMENT 2 NAME"
        },
        "ENVIRONMENT 2 NAME": {
            "cpu_max_threshold": 50,
            "cpu_min_threshold": 20,
            "max_servers": 3,
            "min_servers": 0,
            "scale_by": 1
        }
    }
}

After the second environment has been burst into, it will be scaled up and down according to its own thresholds (and will also grow if the first environment bursts again).