Catalog

Overview

The Catalog enables admins to create blueprints for controlled and repeatable deployments by end users.

A blueprint can be as simple as a one-click single server build and as complex as a multi-tier load balanced application stack running in multiple environments.

A blueprint may even be composed of only actions, with no servers provisioned at all (ex. creating a new LUN by running a CB plugin that interacts with a storage API). Admins can use this flexibility to implement “X as a Service” or XaaS. For example, one could create blueprints for ordering laptops, filing support tickets in an external tracker, or similar solutions not involving actual VM provisioning. Blueprint managers can have their blueprint actually create a resource object to represent the result of ordering that blueprint, such as a laptop, or make no overall resource (e.g., for filing a support ticket, where there’s no need to keep a record of that object in CB). The types of resources available are defined by CloudBolt Admins on the Admin > Resource Types (XaaS) page.

../_images/catalog.png

Blueprints & Resource Lifecycle

A blueprint defines the build process, discovery/ sync logic, management actions, and teardown process for a resource. Blueprints can also be configured to not create a resource when ordered, in which case they only define which servers to build and/or actions to run.

../_images/multitier-blueprint.png

When a new resource is ordered, build blueprint items may create server tiers, run remote scripts on servers, deploy a pod of containers, enact networking changes (such as creating a virtual network or load balancer), email users, connect with external systems, run CloudBolt plugins, and deploy sub-blueprints that create any type of resource (or none).

Another way for a Resource to be created is to be discovered.

Once a resource is deployed, ongoing management may involve resource actions that users can run. These are defined either on the blueprint Management tab (specific to this blueprint) or in Admin > Resource Actions (general purpose).

Finally, when a resource is deleted, actions may be defined on the blueprint Teardown tab to clean up external resources, remove DNS entries, and so forth.

Managers can duplicate a blueprint to easily create similar versions of it, as well as export and import blueprints for version control or sharing between dev and production CloudBolt environments.

Build Configuration

Server Tiers

Server tiers are blueprint items that deploy one or more servers of a particular role. Usually, the OS build and some other primary attributes such as hostname template are predetermined by the admin. Other attributes can be chosen by end users at order time, within the options and constraints specified by the target environment.

These items can be enabled for a single environment or multiple environments, with different parameter options available for each environment.

The following illustrate the flexibility offered by blueprints with regard to how much choice end users have at install time:

  • A single Windows server for a specific environment, always with two CPUs 4 GB of memory and on the same dev VLAN.
  • A single Ubuntu server to the user’s choice of 10 different environments, where the hardware resources change depending on environment. E.g. the server would have one CPU and 2 GB of memory in a VMware environment and an instance type of t1.micro in an AWS environment.
  • A tier that leaves all of these choices up to the end-user: the OS, environment, the quantity of servers to deploy, the environment, resources, network, etc. This is the type of blueprint used by the New Server link.

Cost Preview

For blueprints with a single Server Tier that has specific environments configured (as opposed to when admins simply allow all capable environments) the order form will show end users a cost comparison chart for each environment. This preview is based on the base cost of each environment choice.

Action Blueprint Items

Actions can be added as items within a blueprint to perform any type of operation that is automatable. These actions can be one of any of the five types supported elsewhere in the product and can run code on servers in specific tiers that have just been deployed (remote script actions), on the CB server itself (CloudBolt Plug-ins), and integrate with external systems.

Specific examples of what action blueprint items can do:

  • Email the monitoring team to let them know there is a new deployed resource that should be monitored
  • Connect to the backup system to automatically enable backups for servers deployed by this blueprint
  • Create a new Azure website (see sample scripts in the CloudBolt Forge)
  • Send a message to your team’s chat room (see sample WebHooks in the CB Forge)

These action blueprint items can provide everything they need at runtime or they can use values from servers in other tiers (ex. the IP of the DB server that was deployed earlier in the blueprint deployment). Actions also can ask the user for inputs at runtime. See Action Input Parameters for more information on action inputs, and Action Context for details on what context is provided to actions.

Note

Only CloudBolt Admins can create or edit CloudBolt Plug-ins, even in the context of a Blueprint. Blueprint managers have more control over the other action types. However, if they are not a CB Admin they will not be able to share any actions or edit actions that are shared.

Blueprint Sub-component Items

New in 7.0

Admins can use the Catalog to define Blueprints that can be included as items in other blueprints. Those sub-component blueprints can be as simple or as complex as needed, as long as they meet the following criteria:

  • The sub-blueprints are configured so ANY group can deploy them.
  • The sub-blueprints don’t contain any sub-blueprints.

Note that because there can only be one level of sub-blueprints, a blueprint that is already used as a sub-blueprint will not present the option to add sub-blueprints on its build tab.

If an admin would like to make a particular catalog blueprint available only as a sub-component, it is possible to do so by setting the ‘Is Orderable’ setting on the Blueprint to false.

Blueprint-level Parameters

Blueprint managers have the ability to associate parameters with blueprints using the Parameters tab when managing the blueprint. These parameters will appear when ordering the blueprint, and the values entered will be associated with each build item in the deployed resource, the deployed resource itself, or both, depending on the destination that the manager selects. For example, this could be used to easily set the same Expiration Date for all servers provisioned by the blueprint or to mark the entire resource as being associated with a particular ticket number.

On the Parameters tab when managing the blueprint, it is possible to set options and constraints for a parameter at the blueprint level, just like you can do for Environments and Groups. These options and constraints, however, only apply to the particular blueprint and take precedence over any options or constraints set somewhere else.

If a blueprint-level parameter includes build items as a destination, it will not appear on the form when configuring or ordering individual build items in that blueprint. Note that a preconfiguration will still show on a server blueprint item even if it includes a parameter set at the blueprint level. This is because the preconfiguration likely includes other parameters as well. The blueprint-level value will override the preconfiguration value for that one parameter. For example, if 3 is entered for a blueprint-level Mem Size but small is selected for a Server Size preconfiguration (which translates to 1 CPU and 1 GB memory), the resulting server(s) will have 1 CPU and 3 GB memory.

Resource Names

If desired, Blueprint managers can configure a resource name template that will be used to determine the name of the deployed resource when a blueprint is ordered. Without this setting, the default is to simply use the blueprint name with a unique number. To assign a resource name template to a blueprint, click the Edit button when managing the blueprint. The context available to this template is the blueprint itself with all its attributes, blueprint-level parameters, and the group selected for the order. It uses the Django template format and the same uniquification scheme as elsewhere in CB, such as parameter templates.

For example, to name the resource after the group and the value entered for a blueprint-level parameter named ticket_number, enter {{ group.name }}-{{ blueprint.ticket_number }}-00X. When the blueprint is ordered, assuming it’s configured to create a resource, the deployed resource will end up with a name like “Acme-42-003”.

Ordering

End users order blueprints through the Catalog view. They will see all blueprints that they have permission to order.

Catalog Organization

Your catalog may contain dozens or even hundreds of blueprints. To help users find and discover blueprints, there is a toolbar of filters that makes it easy to search by resource types, labels, or OS build. Users can also search for any blueprint by name or description. The catalog can be sorted alphabetically, by sequence (a weight defined by blueprint managers), by popularity, or how recently they have been added.

../_images/catalog-filters.png

Labels are defined by Super Admins and CloudBolt Admins at Admin > Catalog Management, and then assigned to blueprints by admins or blueprint managers on the Overview tab of a blueprint.

Order Form

Once the blueprint has been set up, end users can order it simply by navigating to the catalog in the CloudBolt user interface and clicking on that blueprint.

If the blueprint has all choices predefined, the user will just need to click the “Submit” button. In fact, blueprint items can be configured to only show if needed, and then the user won’t even have to see them at order time if all the choices are predefined. Otherwise, they may be asked to specify group, environment, hardware and software choices, etc.

If rates are enabled, the order form shows a cost preview comparison chart so users can make an educated decision about which environment to deploy to. This is currently supported for blueprints with a single Server Tier that has been configured for specific environments.

CloudBolt Admins can customize the order in which parameters are shown in blueprint order forms. For example, username fields should typically be grouped alongside and before password fields; hardware resource fields should be grouped together; Quantity should come before most other parameters, and so forth. This is managed in Admin > Orders > Parameter Display Sequence.

When the “Submit” button is clicked, a new order will be created and submitted for approval. If you already have a current order with unsubmitted items, it will be unaffected.

If the order is not auto-approved, an approver can edit it before approving. Similarly, if an order is later duplicated, the owner can edit the new order before submitting it. Note that the current configuration of any associated blueprints, environments, etc. will take precedence over the details of the order being edited if they are different. Also note that if a pre-7.4 order is duplicated, the resulting order may not be editable.

Blueprints that do not Create Resources

Blueprints can also be ordered without creating a deployed resource, if and only if they consist entirely of server build items and/or action build items. If the “Resource Type” in the Edit dialog is None, when a user orders the blueprint it will simply provision the defined server build items and/or run the defined action build items, without doing some resource-related steps. Many of the concepts mentioned elsewhere in this document, such as blueprint-level parameters, resource names, and the resource lifecycle, will not apply when the blueprint does not create a resource. This is the type of blueprint used by the New Server link.

Informational Blueprints

Some IT shops use CloudBolt’s Catalog as the primary place users go to request everything. Some of these blueprints are traditional resources that CB builds, but others are things that can only be ordered from other systems. For these, CB admins can set up informational blueprints, that users can find in the catalog, but the description of the blueprint (which can contain HTML) links to another system. To set one of these up, simply create a blueprint with no build items, but a good description and image, and share it with whichever groups should see it. When “ordering” it, blueprint admins will see a message stating that the blueprint has no build items.

Discovery/ Sync Configuration

A Blueprint can be configured to discover and sync resources of the type that it describes. This can be especially useful for bringing brownfield environments under management.

The main component of configuring the discovery/ sync logic for a Blueprint is the Discovery Plug-in. You can also choose whether any existing Resource associated with the Blueprint should be automatically marked Historical if it is not found by the Discovery Plug-in when it runs. Note that the “Sync Resources” Recurring Job must be enabled for this feature to be used.

Note

Because only CloudBolt Admins can create or edit CloudBolt Plug-ins, a Blueprint manager who is not an Admin will only be able to select an existing shared plug-in to set as a Discovery Plug-in.

Writing a Discovery Plug-in

Some of the details of your Discovery Plug-in will certainly vary based on its purpose and the external system that it connects to, but some of the structure is prescribed.

Planning:

  • For the type of Resources you want to discover and sync, determine which of their attributes should be used to uniquely identify them. Note that this identifier only needs to be unique across Resources associated with this particular Blueprint.
  • Then, determine all the other attributes you would like to store about them, including what type of value each has.
  • Consider permissions, namely what Group should own the discovered Resources and who their owner should be, if anyone.

Requirements:

  • The plug-in must have a global variable RESOURCE_IDENTIFIER defined, whose value is a string matching the key in the discovered dictionaries whose value will be used to uniquely identify Resources (within the Resources associated with that Blueprint).
  • The plug-in must contain a method called discover_resources that returns a list of dictionaries, where each dictionary represents a discovered resource.
  • All keys in those dictionaries should map to attributes that can be set on a Resource object. This includes Resource model fields and Parameters (using their name as the key). If a key does not match a field or current Parameter, a new Parameter will be created with that name. The label will be derived from the name. If the value for that key is an integer, decimal/ float, datetime object, or boolean, the Parameter will be of that type; otherwise, it will default to a string type. If the type created is not what you intended, delete that Parameter and create a new one that matches your needs. All other Parameter attributes will be set to their defaults.

Special cases for setting attributes:

  • Some Resource model fields are always set by CloudBolt, and will be ignored if they are included in a discovered dictionary. These include the Blueprint the Resource is associated with (always the one with the Discovery Plug-in), the Resource Type (always the one associated with the Blueprint), and the jobs associated with the Resource.
  • Some Resource model fields can be provided by the discovered dictionary, but will be set to default values if they are not. If no Group is provided, the “Unassigned” Group will be used. If a name is not provided, the value of the Resource identifier will be used. If no valid value is provided for the lifecycle or a created date & time, they will default to Active and the current time, respectively.
  • If you wish to set either the owner or group of the Resource, find and return a UserProfile or Group object, respectively, in the discovered dictionary.
  • Whenever an invalid value is provided to an attribute, it will not be set. This includes everything from the wrong type of value for a Parameter to an incorrectly-formatted DateTime for the created field to something that is not actually a Group object. The values in the discovered dictionary should be the desired type.

Below is a skeleton example of a possible Discovery Plug-in:

from datetime import datetime

RESOURCE_IDENTIFIER = 'my_id_key'

def discover_resources(*args, **kwargs):
  objects_from_system = <reach out to external system here>
  owner = UserProfile.objects.get(user__username='my username')
  group = Group.objects.get(name='my group name')

  dictionaries = []
  for object_from_system in objects_from_system:
    dictionaries.append({
      'name': object_from_system.name,
      'my_id_key': object_from_system.uuid,
      'owner': owner,
      'group': group,
      'some_attribute': object_from_system.some_attribute,
      # To create the Parameter as a Date & Time type, the value should
      # be a datetime object
      'last_synced': datetime.now(),
      # To create the Parameter as a Boolean type, the value should
      # be a boolean constant object
      'synced_from_system': True,
   })
 return dictionaries

Exporting and Importing Blueprints

Blueprints can be exported as files in order to be easily reused across CloudBolt instances or shared in the CloudBolt Content Library, CloudBolt Forge, a repository of sample CloudBolt content. Exported blueprints are typically stored as zip files containing JSON-formatted metadata about the blueprint along with related items and actions.

It’s possible to export both from the details page of a blueprint in the UI and using the API. When exporting, you can choose between a sanitized or instance-specific version of the blueprint, where the sanitized version will omit potentially sensitive information or details that may not make sense when importing into a different CloudBolt, such as environments and groups. Blueprints can be imported from the Catalog page or using the API, and have an option to either replace the existing blueprint or create a new one.

Exports contain all build and teardown blueprint items, any discovery configuration, all blueprint-specific resource actions, and all action scripts used by these. Note that unless you choose to replace the existing blueprint, new versions of all actions will be created on import. If you’re importing with the replace existing option, it will look for actions that match the attributes appropriate to that specific type of action and use those if they exist.

Exports will also contain all the necessary details on any blueprint-level parameters and, if exported with instance-specific info, enabled environments for blueprint items along with any predefined parameters. The import process for parameters will look for existing matching parameters to use, and create new ones if none are found. However, import of enabled environments will only happen if an environment with the same name is found. This is due to the more complex, instance-specific, and less user-driven nature of environments. Groups will behave the same way as environments.

Similarly, exports of blueprints that create a Resource will contain all the necessary information on the selected Resource Type. The import process will look for an existing matching Resource Type to use, and create a new one if none is found.

Check out the CloudBolt Forge for some example blueprints.

Note

Because only CloudBolt Admins can create or edit CloudBolt Plug-ins, if a Blueprint manager who is not an Admin tries to import a Blueprint that contains one or more CloudBolt Plug-ins, they will receive a permissions error and not be able to do so. Similarly, because non-CB Admins cannot create or edit shared actions, they will not be able to import a Blueprint that contains a shared action or would replace an existing shared action (because they’ve chosen to replace existing), regardless of the action type. Such users can still import Blueprints that do not contain any CloudBolt Plug-ins and would not create or replace shared actions, and CB Admins can import all Blueprints as long as they also have Blueprint management permissions.