Set up high availability in AWS

Configure high availability for Cloud Management Platform in AWS

7 minute read

Create EC2 instances

Consider creating a key pair (in .ppk format) and a security group before creating instances.

Create a primary instance

Log in to the AWS console and go to EC2.
Select Launch instance.
Search for and select the CloudBolt Cloud Management Platform AMI from the AWS Marketplace.
Select Subscribe on instance launch or Subscribe now.
Provide a name for the instance.
Select an instance type.
Select a key pair or create a new one using the .ppk private key file format.
Under Network settings, select Edit, and choose a subnet and availability zone.
Select a security group or choose to create a new one.
Configure any other details as required, then select Launch instance.

Create a secondary instance

Create a second EC2 instance using the same steps as the primary instance, with the following considerations:

Provide a distinct name.
Select a different availability zone.
Select the same security group.

Configure Cloud Management Platform

Initial platform setup

Perform the following steps on all instances.

Launch the Cloud Management Platform web interface.
Follow the steps in Complete initial setup.

Upgrade the platform

Perform the following steps on all instances.

Locate the latest release’s upgrader file from the Support Portal.
SSH into the instance.

Run the following commands:

sudo su -
cd /tmp
curl latest-release-upgrader-url | tar xvz
cd latest-release-upgrader-name/
./upgrade_cloudbolt.sh

The upgrade can take up to ten minutes to complete.

Prepare for database and file system configuration

Perform the following steps on the primary instance.

Open an SSH session.
Copy the contents of the instance’s private key to a .ppk file (for example, mykey.ppk) in the instance’s tmp directory.
Run the following command to update the key permissions:
```
chmod 600 /tmp/mykey.ppk
```
Run the following command to update permissions on secret-key-for-cloudbolt.bin:
```
chmod 777 /var/opt/cloudbolt/secrets/secret-key-for-cloudbolt.bin
```

Perform the following steps on secondary instances.

Run the following command:

chmod 777 /var/opt/cloudbolt/secrets/secret-key-for-cloudbolt.bin

Copy the bin file from the primary instance.
Check customer_settings.py and verify that UNIQUE_TOKEN matches the primary instance.
```
cat /var/opt/cloudbolt/proserv/customer_settings.py
```
If UNIQUE_TOKEN does not match, copy it from the primary instance.

Perform the following steps on all instances.

Open the job engine supervisor configuration file for editing:
```
vi /etc/supervisord.d/jobengine.conf
```
Set autostart and autorestart to false and save the file.
Run the following command:
```
supervisorctl reload
```

Dump the existing SQL database:

source /var/opt/cloudbolt/.postgresql_password
export PGPASSWORD=${postgresql_pass}
pg_dump -h localhost -p 5432 -U cb_dba -w --clean -F t -f /tmp/cloudbolt.tar cloudbolt

The /var/opt/cloudbolt/.postgresql_password file contains postgresql_root_pass and postgresql_pass.

Create an application load balancer

In the AWS console, go to EC2.
Select Load Balancers.
Select to create an Application Load Balancer.
Provide a name.
Select a VPC, at least two Availability Zones, and a subnet for each zone.
Choose up to five security groups.
Under Listeners and routing > Listener > Protocol, select HTTPS.
Under Routing action > Forward to target groups, choose create target group.
Provide a name, select HTTPS, configure any other required details, and select Next.
Add the primary and secondary EC2 instances as targets and select Next.
Review the details and select Create target group.
Under Secure listener settings, select a security policy and SSL certificate.
Select Create load balancer.

Create and configure an Elastic File System

Create the file system

In the AWS console, go to EFS.
Select Create file system.
Provide an optional name, choose a VPC, and select Customize.
Select Next.
Configure the mount targets, and select Next.
Optionally configure the file system policy, and select Next.
Review the file system configuration and select Create.

Retrieve the DNS name

Select the file system name.
Select Attach, note the file system’s DNS name for later use, and close the Attach modal.

Configure the instances to use the file system

Open an SSH session to each instance.
Run the following command to install NFS:
```
sudo yum install nfs-utils
```

Run the following commands, replacing dns-name with the DNS name noted previously:

mkdir /mnt/efs
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport dns-name:/ /mnt/efs

On the primary instance only, run the following commands:

mv /var/opt/cloudbolt /mnt/efs/cloudbolt
mv /var/log/cloudbolt/jobs /mnt/efs/jobs
mv /var/www/html/cloudbolt/static /mnt/efs/static
mkdir /var/opt/cloudbolt
mkdir /var/log/cloudbolt/jobs
mkdir /var/www/html/cloudbolt/static
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport dns-name:/cloudbolt /var/opt/cloudbolt
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport dns-name:/jobs /var/log/cloudbolt/jobs
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport dns-name:/static /var/www/html/cloudbolt/static

On secondary instances only, run the following commands:

rm -rf /var/opt/cloudbolt
rm -rf /var/log/cloudbolt/jobs
rm -rf /var/www/html/cloudbolt/static
mkdir /var/opt/cloudbolt
mkdir /var/log/cloudbolt/jobs
mkdir /var/www/html/cloudbolt/static
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport dns-name:/cloudbolt /var/opt/cloudbolt
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport dns-name:/jobs /var/log/cloudbolt/jobs
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport dns-name:/static /var/www/html/cloudbolt/static

On all instances, open the /etc/fstab file for editing, add the following lines, and save the file:

dns-name:/cloudbolt /var/opt/cloudbolt nfs rw,hard,intr,rsize=8192,wsize=8192,timeo=14 0 0
dns-name:/jobs /var/log/cloudbolt/jobs nfs rw,hard,intr,rsize=8192,wsize=8192,timeo=14 0 0
dns-name:/static /var/www/html/cloudbolt/static nfs rw,hard,intr,rsize=8192,wsize=8192,timeo=14 0 0

On all instances, run mount -fav to validate the update.

Create and configure an RDS database

When configuring an RDS database, decide between PostgreSQL and Aurora, and follow the corresponding steps.

Create an RDS PostgreSQL database

In the AWS console, go to Aurora and RDS.
Select Create a database.
Select PostgreSQL.
Select an engine version with the same major version as the PostgreSQL database on your Cloud Management Platform (CMP) instances, so the backup restores cleanly. To check it, run sudo -u postgres psql -Atc "show server_version" on the primary instance.
Select the production template.
Under Settings, provide an instance identifier, master username, and select a credentials management option.
Under Connectivity, select a VPC, subnet group, and security group.
Expand Additional configuration and provide an initial database name.
Configure any additional parameters as required, then select Create database.

Create an RDS Aurora database

In the AWS console, go to Aurora and RDS.
Select Create a database.
Select Aurora (PostgreSQL Compatible).
Select a version with the same major version as the PostgreSQL database on your CMP instances.
Select the production or dev/test template.
Under Settings, provide a cluster identifier and master username, then select a credentials management option.
Configure scaling and select a redundancy option.
Under Connectivity, select a VPC, subnet group, and security group.
Configure any additional parameters as required, and select Create database.

Select the database, open the configuration tab, and copy the endpoint address.

Configure the instances to use the database

Perform the following steps on the primary instance. In each command, replace db-dns-name with the database’s endpoint address, username and password with the master user’s credentials, and database-name with the database name.

Open an SSH session.

Create the database. Skip this step if you set an initial database name when you created the RDS database, and use that name as database-name.

export PGPASSWORD='password'
psql --host=db-dns-name --port=5432 --username=username --dbname=postgres -c "create database database-name;"

Restore the database backup to the new database:

export PGPASSWORD='password'
pg_restore -h db-dns-name -p 5432 -U username --no-owner --role=username -w -e --clean --if-exists -d database-name /tmp/cloudbolt.tar

Set the search path for the database user:

psql "postgresql://username:password@db-dns-name:5432/database-name" -c "alter role username in database database-name set search_path to cloudbolt,public;"

Verify the restore. The command returns the number of CMP tables, which should be greater than 0.

psql "postgresql://username:password@db-dns-name:5432/database-name" -Atc "select count(*) from information_schema.tables where table_schema='cloudbolt';"

Copy the DATABASES section from /opt/cloudbolt/settings_local.py to /var/opt/cloudbolt/proserv/customer_settings.py, updating the NAME, USER, PASSWORD, and HOST fields to the RDS database’s values.

Perform the following steps on all instances.

Run the following command to stop and disable PostgreSQL. The service is postgresql-18 on nodes installed with CMP 2026.3.97 or later, and postgresql-14 on older nodes.
```
systemctl stop postgresql-18 && systemctl disable postgresql-18
```
Run the following command to restart httpd:
```
systemctl restart httpd
```
Open the job engine supervisord configuration file located at /etc/supervisord.d/jobengine.conf for editing, set autostart and autorestart to true, and save the file.
Run sudo reboot to reboot the instances.
Confirm you can log in to the rebooted instances.