Resources

A resource is the representation in CloudBolt of some item or group or items in your infrastructure. Resource is the base type of object, but you can choose to configure different types of Resources, and a Resource will largely be referred to using its type. A Resource could represent anything from a set of VMs that together constitute a web application to a single S3 bucket.

The creation and management of Resources is defined by Blueprints. When a Blueprint is ordered in CloudBolt, a Resource may be created based on the configuration of its Build tab. Resources may also be created, and later updated, by the Discovery logic that is provided for a Blueprint.

Once a resource has been created, it can be managed in the CloudBolt user interface, including scaling it up or down, executing arbitrary actions on it, and eventually deleting it.

Syncing Resources

In addition to creating new discovered resources, the Discovery Plug-in associated with a Blueprint can also update attributes of existing resources associated with that Blueprint. For example, if a resource already has a value of 42 for the parameter Available Storage when the Recurring Job to Sync Resources runs, and the discovered dictionary identified to match with the resource now returns a value of 24 for that attribute, the parameter’s value on the resource will be changed to 24.

There is a Recurring Job named Sync Resources that comes disabled out-of-the-box with a once-per-day schedule. To make use of the Discovery/ Sync feature, enable this job and adjust the schedule if needed.

Whenever the job runs, it will check for Blueprints with a configured Discovery Plug-in and run that logic for each. Then, the list of dictionaries returned by each Discovery Plug-in will be examined and compared to existing Resources associated with the Blueprint under consideration. The value of the attribute specified as the Resource identifier will be used to match the discovered dictionaries to Resources. If a Resource matches, it may be updated. If there is no matching Resource, a new one will be created based on the discovered information. For more details, see the section on writing a Discovery Plug-in.

The job will be considered a failure if sync fails entirely for any 1 (or more) Blueprint, but it will not prevent the Resources for other Blueprints from being synced. It is also possible for only individual dictionaries or keys for a sync to be skipped, which will be logged but not cause the job to fail.

Automatically Marking Resources Historical

One optional feature when syncing existing Resources is to mark them Historical when they are not found by the sync logic. The assumption is that if they aren’t found then they no longer exist, and should therefore have their lifecycle updated to indicate that they have been deleted. This will impact where the Resource is visible in the CloudBolt UI.

If enabled, the sync logic will check for any existing Resources associated with the Blueprint being synced whose identifier value did not appear in any of the discovered dictionaries returned by the Discovery Plug-in. Any such Resources will become Historical.

Resource Actions

Resource Actions are a mechanism for adding arbitrary actions to deployed resources, via buttons on their details view. For any action that end-users may need to invoke on deployed resources, CloudBolt admins can add a resource action to appear on the deployed resource’s detail page and, when clicked, run one of the five types of CloudBolt actions.

These actions can run scripts on all or a subset of server tiers in the resource, contact external systems, email users, and run code that interacts with the CB database to read and store information on the resource. For examples of resource actions, check the CloudBolt forge or ask your CloudBolt System Engineer.

Actions can be restricted to particular blueprints, so that they will only be shown on resources deployed from those blueprints. For example, a “Restart Web Server” action could be created and associated with the web server blueprint, and that button would only appear on resources deployed from that blueprint.

Creating Attributes on Resources

CloudBolt administrators have the ability to add arbitrary name/value pairs to the details page for deployed resources. For example, when deploying a website resource admins may want to store and display the port that website is listening on, or any other data the user may need to interact with the site.

One way this can be achieved is through the use of Blueprint-level Parameters that have a destination of the deployed resource. It can also be done through a plug-in action build item on a Blueprint that runs during deployment. These plug-ins can associate arbitrary name value pairs with deployed resources and they will be seen by users viewing the overview tab of the resource. There are examples of associating this metadata with deployed resources in the CloudBolt forge.

Auto Scaling

Deployed resources can be configured so that their tiers scale based on CPU load (and other conditions).

Overview

CloudBolt can run actions to detect CPU load on a tier of servers in a deployed resource, and if a maximum threshold is reached, it can initiate the building of new servers to add to that tier and then update the load balancer to add the new servers to the rotation. Later, when CloudBolt detects that the load is below a minimum threshold, it can delete servers in the tier, automatically removing them from the load balancer rotation as it does so.

The auto scaling feature has been built using CloudBolt’s rules engine, with its central logic contained in CloudBolt Plugins so that the behavior is transparent and extensible. CloudBolt customers can override this logic if they would like auto scaling to behave differently than it does out of the box.

Prerequisites

Before beginning with auto scaling, build a simple blueprint that consists of one or more servers and optionally a load balancer. A blueprint that deploys two simple web servers with some static content and a load balancer directing traffic to the two servers would be ideal. Then deploy this blueprint, ensure the resultant resource is behaving as expected, and that running manual scale up and scale down operations from the CloudBolt UI works properly.

How to Configure Auto Scaling

  1. To enable the auto-scaling rule and add the actions and parameters necessary for this, ssh to the CloudBolt server as root and run /opt/cloudbolt/initialize/create_objects.py /opt/cloudbolt/initialize/cloud_bursting.py
  2. In the CB UI, navigate to Admin > Actions and filter to actions that contain “Get CPU”.
  3. ^Click on “Get CPU Utilization for VMware”, edit it, and restrict it to the VMware resource technology
  4. ^Go back to the list of actions, click on “Get CPU Utilization for AWS”, edit it, and restrict it to the AWS resource technology
  5. In the CB UI, navigate to the resource you would like to auto scale, click the parameters tab, and add the parameter “Auto-Scaling Config”. Here is a sample auto-scaling configuration value to use (replacing the capitalized sections):
{
    "TIER NAME": {
        "ENVIRONMENT NAME": {
            "cpu_max_threshold": 40,
            "cpu_min_threshold": 10,
            "max_servers": 5,
            "min_servers": 1,
            "scale_by": 1
        }
    }
}

^-this step will not be necessary in future versions of CloudBolt

How to Test

First, test scaling up:

  1. Add the “Simulated CPU Load” parameter to the resource, giving it a value of 50
  2. Open a separate tab in your browser and navigate to Admin > Rules
  3. Run the “Check Resources for Auto-Scaling Conditions” Rule
  4. Click on the job that was created. It should detect that the load is beyond the threshold and initiate a job to provision another web server
  5. While the provisioning job is running, return to your browser tab with the resource details. Click on the overview tab for the resource and note the depiction of a new server being built for the resource
  6. After the provisioning is done, you can confirm that the load balancer has been updated to include the new server

Then test scaling back down:

  1. Change the “Simulated CPU Load” parameter’s value to 5
  2. Run the “Check Resources for Auto-Scaling Conditions” Rule again
  3. Click on the new job. This check should initiate a scale down job to delete one of the servers in the resource

Then, to clean up, remove the “Simulated CPU Load” parameter from the resource.

Lastly, test load detection with real CPU load:

  1. Generate significant CPU load on the servers in the scaling tier
  2. Run the rule
  3. Ensure that the rule scales the tier up
  4. Reduce the load
  5. Run the rule
  6. Ensure that the rule scales the tier back down

Bursting Configuration

In order to cause a tier to scale up into a different environment:

  1. In the blueprint, configure that server blueprint item to be deployed into another environment
  2. In a deployed resource, in the Auto-Scaling Config, specify a second environment for the tier
  3. Add a “burst_into” key in the first environment to cause it to scale into the second environment after the first is at its max
{
    "TIER NAME": {
        "ENVIRONMENT 1 NAME": {
            "cpu_max_threshold": 20,
            "cpu_min_threshold": 10,
            "max_servers": 2,
            "min_servers": 1,
            "scale_by": 1,
            "burst_into": "ENVIRONMENT 2 NAME"
        },
        "ENVIRONMENT 2 NAME": {
            "cpu_max_threshold": 50,
            "cpu_min_threshold": 20,
            "max_servers": 3,
            "min_servers": 0,
            "scale_by": 1
        }
    }
}

After the second environment has been burst into, it will be scaled up and down according to its own thresholds (and will also grow if the first environment bursts again).