Set up high availability

Create the required number of Cloud Management Platform (CMP) instances. You can configure high availability on-premises, in AWS, or in Azure.

Configuration overview

At a high level, the following tasks are required to configure high availability for CMP.

1. Install CMP nodes as required.
2. License the nodes using the CMP web interface.
3. If required, upgrade all nodes using the CLI.
4. Database tasks:
   1. Deploy an external shared database.
   2. Disable the built-in database server.
   3. Update all nodes to reference the external database.
5. NFS tasks:
   1. Move node files to NFS.
   2. Delete local files from each node.
   3. Update all nodes to reference NFS.
6. Disable Web Server/Job Engine roles as necessary.
7. Configure the load balancer.

Web server

CMP uses the Apache HTTP Server, which runs as a httpd process on the appliance.

Configure web servers

To create a web server, clone an existing CMP web server, or run the CMP installer on a new VM.

Ensure each web server has access to the required shared directories, as described in the reference architecture.

Then perform the following steps:

1. Disable the job engine service
2. Disable PostgreSQL
3. Configure connection to external database

1: Disable job engine

The instance does not require the job engine services.

1. Open the job engine supervisord configuration file located at /etc/supervisord.d/jobengine.conf for editing.

2. Set autostart and autorestart to false, and save the file.

3. Run the following command to reload the configuration files.

   Use sudo su - before running supervisorctl to access elevated permissions.

   supervisorctl reload
   

2: Disable PostgreSQL

The instance does not require PostgreSQL.

Run the following command to stop PostgreSQL and prevent it from starting during instance reboot. The service is postgresql-18 on nodes installed with CMP 2026.3.97 or later, and postgresql-14 on older nodes.

systemctl stop postgresql-18 && systemctl disable postgresql-18

3: Configure connection to external database

Update the database settings to point to the external database.

1. Open settings_local.py and copy the DATABASES section.

2. Open customer_settings.py for editing, and paste the DATABASES section.

3. Update the HOST, USER, and PASSWORD values in the DATABASES section to align with your database:

   HOST: <database-ip-or-fqdn>
   USER: <database-username>
   PASSWORD: <database-password>
   

Job engine

CMP runs its job engine through supervisor, which can be controlled through supervisorctl commands.

Manage job engine processes by editing the configuration file at /etc/supervisord.d/jobengine.conf.

1. Ensure there are no jobs running.

2. Stop all job engine processes.

   supervisorctl stop jobengine:*
   

Configure job engine servers

To create a job engine server, clone an existing CMP job engine server, or run the CMP installer on a new VM.

Ensure each job engine has access to the required shared directories, as described in the reference architecture.

Then perform the following steps:

1. Disable the web service
2. Disable PostgreSQL
3. Configure connection to external database
4. Disable orphaned job cleanup, if you run more than one job engine server

1: Disable the web service

The instance does not require Apache.

Run the following command to stop the web server process and prevent it from starting during instance reboot.

systemctl stop httpd && systemctl disable httpd

2: Disable PostgreSQL

The instance does not require PostgreSQL.

Run the following command to stop PostgreSQL and prevent it from starting during instance reboot. The service is postgresql-18 on nodes installed with CMP 2026.3.97 or later, and postgresql-14 on older nodes.

systemctl stop postgresql-18 && systemctl disable postgresql-18

3: Configure connection to external database

Update the database settings to point to the external database.

1. Open settings_local.py and copy the DATABASES section.

2. Open customer_settings.py for editing, and paste the DATABASES section.

3. Update the HOST, USER, and PASSWORD values in the DATABASES section to align with your database:

   HOST: <database-ip-or-fqdn>
   USER: <database-username>
   PASSWORD: <database-password>
   

4: Disable orphaned job cleanup

Do this only if you run more than one job engine server. With several engines, the orphaned job cleanup can cancel jobs that another engine is still running. With the cleanup disabled, a job whose worker has died stays running until someone cancels it.

1. Disable the feature. The setting is stored in the shared database, so run this once, from any node:

   /opt/cloudbolt/manage.py features --disable jobengine::cancel_orphan_jobs
   

2. So that an upgrade does not turn the cleanup back on, add the following to customer_settings.py on each job engine server and save the file:

   from settings import FEATURE_REGISTRY
   
   FEATURE_REGISTRY["jobengine::cancel_orphan_jobs"] = False
   

3. Confirm the change. Job engine workers pick it up within about a minute.

   /opt/cloudbolt/manage.py features --list
   

Database

You can set up a PostgreSQL cluster or replication topology as desired to support CMP. For example, PostgreSQL database servers can be configured to support one-way replication or configured as a redundant cluster.

Refer to the PostgreSQL documentation for more information on configuring replication. To upgrade a CMP-managed database from PostgreSQL 14, see Upgrade to PostgreSQL 18.

If you are unfamiliar with PostgreSQL administration, we recommend using a hosted PostgreSQL service.