Set up high availability in Azure

Configure high availability for Cloud Management Platform in Azure

8 minute read

A high availability (HA) deployment of Cloud Management Platform (CMP) in Azure uses two or more VM nodes behind a Standard Azure Load Balancer, with a shared Azure Database for PostgreSQL flexible server and an Azure Files NFS share providing a common data layer across all nodes.

Create virtual machines

Deploy in the region closest to your physical infrastructure, or the region where the majority of your cloud infrastructure resides. Before creating virtual machines, ensure that Azure Database for PostgreSQL flexible server is available in your desired region.

Create the required number of Azure virtual machines, such as one primary and one secondary, with either an Oracle Enterprise Linux 8 or RHEL 8 image.

Configure Cloud Management Platform

Perform the following steps on all instances.

Deploy Cloud Management Platform

Deploy using the installer.

Initial platform setup

Perform the following steps on all instances.

Launch the CMP web interface.
Follow the steps in Complete initial setup.

Configure node connectivity

Remove any public IPs created during deployment.
Ensure you can connect to the instances, for example, using Azure Bastion, Azure Cloud Shell, or another VM.

Upgrade the platform

The upgrade can take up to ten minutes to complete.

Locate the latest release’s upgrader file from the Support Portal.
SSH into the instance.

Run the following commands, replacing latest-release-upgrader-url with the URL of the upgrader file:

sudo su -
wget latest-release-upgrader-url
tar -xvzf cloudbolt_upgrader_*.tgz
cd cloudbolt_upgrader_*
./upgrade_cloudbolt.sh

Prepare for database configuration

Perform the following steps on the primary instance.

The /var/opt/cloudbolt/.postgresql_password file contains postgresql_root_pass and postgresql_pass.

Open an SSH session.

Make a copy of the instance’s database:

source /var/opt/cloudbolt/.postgresql_password
export PGPASSWORD=${postgresql_pass}
pg_dump -h localhost -p 5432 -U cb_dba -w --clean -F t -f /tmp/cloudbolt.tar cloudbolt

Run the following command to update permissions on secret-key-for-cloudbolt.bin:
```
chmod 777 /var/opt/cloudbolt/secrets/secret-key-for-cloudbolt.bin
```

Create and configure a PostgreSQL database

Create an Azure Database for PostgreSQL flexible server

In the Azure portal, go to Azure Database for PostgreSQL flexible servers and select Create.
Select a Subscription and Resource Group.
Provide a Server name.
Select the same region as your VMs.
Select PostgreSQL version 18.
For Workload type, select Dev/Test or Production.
Configure high availability if required.
Under Authentication, provide Administrator login: cb_dba and a password.
Select Next : Networking >.
Select Private Access (VNet Integration) under Connectivity method.
Under Virtual network select a Virtual network that has a subnet that is delegated for use for PostgreSQL Flexible Server, or create a new one.
Select Review + Create.
Select Create.

After creating the database, select it and copy the Endpoint FQDN from the overview tab.

Configure the instances to use the database

Perform the following steps on the primary instance.

Create the CMP database in the PostgreSQL database, replacing remote-db-dns-name, remote-db-user, and remote-db-pass:

export PGPASSWORD='<remote-db-pass>'
psql --host=remote-db-dns-name --port=5432 --username=remote-db-user --dbname=postgres <<END_OF_SQL
drop database if exists cloudbolt with (force);
create database cloudbolt;
grant all privileges on database cloudbolt to cb_dba;
END_OF_SQL

Restore the database copy to the new PostgreSQL database, replacing remote-db-dns-name, remote-db-user, remote-db-pass, and remote-db-port:

export PGPASSWORD='<remote-db-pass>'
pg_restore -h remote-db-dns-name -U remote-db-user --no-owner --role=remote-db-user -w -p remote-db-port -e --clean --if-exists -d cloudbolt /tmp/cloudbolt.tar

Run the following commands to set permissions for cb_dba, replacing remote-db-dns-name, remote-db-user, and remote-db-pass:

export PGPASSWORD='<remote-db-pass>'
psql --host=remote-db-dns-name --port=5432 --username=remote-db-user --dbname=postgres <<END_OF_SQL
alter role cb_dba in database cloudbolt set search_path to cloudbolt,public;
grant all privileges on database cloudbolt to cb_dba;
END_OF_SQL

Run the following commands to verify the database connection, ensuring they return values, replacing remote-db-dns-name, remote-db-user, remote-db-pass, and remote-db-port:
```
psql postgresql://remote-db-user:remote-db-pass@remote-db-dns-name:remote-db-port/cloudbolt -c "\l" | grep -qw cloudbolt
```
Copy the DATABASES section from /opt/cloudbolt/settings_local.py to /var/opt/cloudbolt/proserv/customer_settings.py, updating the password and hosts fields.

Perform the following steps on all instances.

Run the following commands to stop and disable PostgreSQL:
```
systemctl stop postgresql-18; systemctl disable postgresql-18
```
The service is postgresql-18 on nodes installed with CMP 2026.3.97 or later, and postgresql-14 on older nodes.

If the AllowNfsFileShares feature is already registered, skip this step and continue to Create a storage account and file share.

az feature register --name AllowNfsFileShares --namespace Microsoft.Storage

Verify feature registration:

az feature show --name AllowNfsFileShares --namespace Microsoft.Storage --query properties.state

resourceGroup="<resource-group>"
storageAccountName="<orgname>cmpfiles"
location="<region>"

az storage account create \
    --resource-group $resourceGroup \
    --name $storageAccountName \
    --location $location \
    --sku Premium_LRS \
    --kind FileStorage

az storage share-rm create \
    --resource-group $resourceGroup \
    --storage-account $storageAccountName \
    --name nfscmpfiles \
    --enabled-protocol NFS \
    --root-squash NoRootSquash \
    --quota 100

$resourceGroup = "<resource-group>"
$storageAccountName = "<orgname>cmpfiles"
$location = "<region>"

az storage account create `
    --resource-group $resourceGroup `
    --name $storageAccountName `
    --location $location `
    --sku Premium_LRS `
    --kind FileStorage

az storage share-rm create `
    --resource-group $resourceGroup `
    --storage-account $storageAccountName `
    --name nfscmpfiles `
    --enabled-protocol NFS `
    --root-squash NoRootSquash `
    --quota 100

Configure the storage account

In the Azure portal, go to Storage accounts and select the new storage account.
Under Properties, select Networking.
Under Public network access, select Manage.
Next to Public network access, select Enable.
Next to Public network access scope, select Enable from selected networks.
Select the virtual networks, IP addresses, and resource instances that can access the account, and select Save.
Under Settings, select Configuration.
Set Secure transfer required to Disabled, and select Save.
Under Data storage, select File shares.
Select the file share.
Select Connect.
Select Linux.
Copy the Mount point.

Configure NFS on the instances

SSH into all instances.

On all instances, run the following commands, using the mount path:

sudo yum install nfs-utils
mkdir /mnt/nfs
sudo mount -t nfs -o vers=4,minorversion=1,sec=sys <storageaccount>.file.core.windows.net:/<storageaccount>/nfscmpfiles /mnt/nfs

On the primary instance only, run the following commands:

mv /var/opt/cloudbolt /mnt/nfs/cloudbolt
mv /var/log/cloudbolt/jobs /mnt/nfs/jobs
mv /var/www/html/cloudbolt/static /mnt/nfs/static

On secondary instances only, run the following commands:

rm -rf /var/opt/cloudbolt
rm -rf /var/log/cloudbolt/jobs
rm -rf /var/www/html/cloudbolt/static

On all instances, run the following commands:

mkdir /var/opt/cloudbolt
mkdir /var/log/cloudbolt/jobs
mkdir /var/www/html/cloudbolt/static
sudo mount -t nfs -o vers=4,minorversion=1,sec=sys <storageaccount>.file.core.windows.net:/<storageaccount>/nfscmpfiles/cloudbolt /var/opt/cloudbolt
sudo mount -t nfs -o vers=4,minorversion=1,sec=sys <storageaccount>.file.core.windows.net:/<storageaccount>/nfscmpfiles/jobs /var/log/cloudbolt/jobs
sudo mount -t nfs -o vers=4,minorversion=1,sec=sys <storageaccount>.file.core.windows.net:/<storageaccount>/nfscmpfiles/static /var/www/html/cloudbolt/static

On all instances, open the /etc/fstab file for editing, add the following lines (using the mount path), and save the file:

<storageaccount>.file.core.windows.net:/<storageaccount>/nfscmpfiles/cloudbolt /var/opt/cloudbolt nfs rw,hard,intr,rsize=8192,wsize=8192,timeo=14 0 0
<storageaccount>.file.core.windows.net:/<storageaccount>/nfscmpfiles/jobs /var/log/cloudbolt/jobs nfs rw,hard,intr,rsize=8192,wsize=8192,timeo=14 0 0
<storageaccount>.file.core.windows.net:/<storageaccount>/nfscmpfiles/static /var/www/html/cloudbolt/static nfs rw,hard,intr,rsize=8192,wsize=8192,timeo=14 0 0

On all instances, run mount -fav to validate the update.

Reboot instances

On all instances, run sudo reboot to reboot the instances.

Post-reboot checks

After rebooting, verify that essential services are starting and healthy on each of your nodes.

Apache HTTP server

Check the httpd status:
```
systemctl status httpd
```
If httpd fails to start or is unhealthy, you may need to set SELinux to permissive mode, or create SELinux entries to grant httpd access to NFS share files.

Option 1: Run SELinux in permissive mode
1. Set SELinux to permissive for the current session:
```
setenforce 0
```
2. Open /etc/selinux/config for editing:
```
vi /etc/selinux/config
```
3. Set SELINUX to permissive and save the file:
```
SELINUX=permissive
```
4. Restart the CMP application:
```
supervisorctl restart jobengine:* && systemctl restart httpd
```
Option 2: Add SELinux entries
1. Refer to How to Prepare SELinux Policies for CMP Appservers

httpd should now be starting and healthy. Ensure there are no errors in /var/log/cloudbolt/application.log. Load the CMP UI.

Firewall

If httpd is healthy and you are unable to access the CMP UI, ensure the firewall is configured to allow access on the required ports. The following instructions are for firewalld:
```
firewall-cmd --permanent --add-service=http
firewall-cmd --permanent --add-service=https
firewall-cmd --reload
```

CMP system status

Check System Status to ensure all services are green:

In the CMP UI, go to Admin > All Admin > System Status.
Under Health Checks, verify that all services are green.

Create an Azure Load Balancer

These instructions document how to create an Azure Load Balancer that enables SSL termination on each CMP instance rather than on the load balancer. If you require more advanced load balancing capabilities (for example, SSL termination at the load balancer), contact your Technical Account Manager for assistance with configuring an Azure Application Gateway.

In the Azure portal, go to Load balancers.
Select Create > Standard Load Balancer.
Select a Subscription and Resource Group.
Provide a Name.
Select the same Region as your VMs.
Next to Type, select Public.
Select Next : Frontend IP Configuration >.
1. Select Add a frontend IP configuration.
  1. Provide Name: cmp-vip, select to create a new public IP address, and select Save.
Select Next : Backend pools >.
1. Select Add a backend pool.
  1. Provide Name: cmp-pool, select the instance virtual network, add the instances, and select Save.
Select Next : Inbound rules >.
- Select Add a load balancing rule to create a rule for HTTP.
  - Frontend IP address: cmp-vip
  - Backend pool: cmp-pool
  - Port: 80
  - Backend port: 80
  - Health probe: Create new
    - Name: cmp-check
    - Protocol: TCP
    - Port: 80
    - Select Save.
  - Session persistence: Source IP
  - Select Save.
- Select Add a load balancing rule to create a rule for HTTPS.
  - Name: https
  - Frontend IP address: cmp-vip
  - Backend pool: cmp-pool
  - Port: 443
  - Backend port: 443
  - Health probe: cmp-check
  - Session persistence: Source IP
  - Select Save.
Select Next : Outbound rule >.
- Select Add an outbound rule.
  - Name: cmplb-out
  - Frontend IP address: cmp-vip
  - Backend pool: cmp-pool
  - Port allocation: Use the default number of outbound ports
  - Select Save.
Select Review + create.
Select Create.

Set up high availability in Azure

Create virtual machines

Configure Cloud Management Platform

Deploy Cloud Management Platform

Initial platform setup

Configure node connectivity

Upgrade the platform

Prepare for database configuration

Create and configure a PostgreSQL database

Create an Azure Database for PostgreSQL flexible server

Configure the instances to use the database

Create and configure an Azure Files NFS file share

Register the NFS file share feature

Create a storage account and file share

Configure the storage account

Configure NFS on the instances

Reboot instances

Post-reboot checks

Apache HTTP server

Firewall

CMP system status

Create an Azure Load Balancer