# VMware data collector configuration

> The VMware data collector is a component of the Cloud Management Platform (CMP) appliance that collects inventory of assets for your on-premises infrastructure and pushes them to the CloudBolt CSMP

---

Site [llms.txt](/llms.txt)

---

The collector is deployed with CMP and does not require a CSMP license to run.

The collector can be configured in one of two ways:

1. If you are a CMP-licensed customer, then configuration can be done using the standard CMP Administration UI.
1. If you only have a CSMP license, deploy the CMP appliance in Collector mode and use [management commands](#set-up-with-the-collector_config-command).

<div class="alert alert-info" role="alert">


CMP also collects OpenStack inventory through an **OpenStack Data Collector** recurring job and an `openstack` sub-command. This page covers the VMware collector.
</div>


## Installation and configuration guide

### Installation

Install the appliance as usual by deploying the OVA or running the installer, then complete its network configuration. See [Install Cloud Management Platform](/docs/cmp/get-started/install/).

If you are a CSMP-only customer without a CMP license, you will need to set the appliance mode to "CSMP Collector."

To set the appliance to CSMP Collector mode and the time zone:

1. Add the following to the `/var/opt/cloudbolt/proserv/customer_settings.py` file.

   ```text
   CSMP_DATA_COLLECTOR_ENABLED=True
   TIME_ZONE = 'America/New_York'
   ```

1. Execute the following command to create the required base objects.

   ```text
   /opt/cloudbolt/initialize/create_objects.py /opt/cloudbolt/initialize/cb_minimal.py
   ```

1. Restart the Job Engine.

   ```text
   supervisorctl restart jobengine:*
   ```

## Configure the collector

To set up the data collector:

1. In the CSMP tenant, configure one VMware adaptor and configure the VMware rate card.
1. Configure one Connection Info named **CSMP Data Collector** with your CSMP Domain URL and CSMP API Key. The collector looks for that exact name and uploads nothing to CSMP without it.
1. Configure one or more resource handlers in CMP for the CSMP adaptors to collect data from.
1. For each resource handler, create a recurring job from the **CSMP VMware Data Collector** action, scheduled to run every 30 minutes. The `collector_config vmware create` command below does this for you.

<div class="alert alert-info" role="alert">


This collector supports a **proxy server**. The resource handler's service account must have the **built-in Read-Only** role and **Storage Profile permissions** to collect data successfully.
</div>


## Set up with the collector_config command

If you are a CSMP-only customer, you can configure the above steps using the collector_config management command.

1. To create the **CSMP Data Collector** Connection Info with your CSMP Domain URL and CSMP API Key, run this command:

   ```text
   /opt/cloudbolt/manage.py collector_config csmp create --domain-url https://mycsmp.cloudbolt.app --api-key yourapikey
   ```

   Fetch the VMware adapter key from the CSMP portal as follows:

   1. Log in to the CSMP portal, for example `https://abcxys.cloudbolt.app/`, and go to VMware Adapters.
   1. Generate the Access key for the specific VMware Adapter as shown in the following screenshot:

      ![Screenshot of the CSMP Cloud Adapters page, with the Generate Access Keys option in a VMware adapter's Action menu.](/img/cmp/configure/integrations/cost-and-security-management/vmware-data-collector/vmware-adapter-access-keys.png)

   1. Select **Generate Access Keys** to generate the required API key.

1. To configure resource handlers in CMP, run this command:

   ```text
   /opt/cloudbolt/manage.py collector_config vmware create --host myvcenter.cloudbolt.com --username yourvcenteruser --password yourvcenterpassword
   ```

That command takes the same arguments as the **vmware** sub-command's [create](#vmware) operation.

The **vmware create** command also creates the resource handler's recurring job. To confirm it exists, run this command:

```text
/opt/cloudbolt/manage.py collector_config recurring-job list
```

Sample Output

```text
+--------------+--------------------------------------------------------------------+---------+----------------------+
|      ID      |                                Name                                | Enabled |       Last Run       |
+--------------+--------------------------------------------------------------------+---------+----------------------+
| RJB-374fv8ip | CSMP VMware Data Collector - vcenter2-csmp.cblabqa.net-RH-ly17mzoa |   True  | 01-Apr-2024 18:30:00 |
+--------------+--------------------------------------------------------------------+---------+----------------------+
```

## collector_config command reference

The section below describes how to use the collector_config management command.

The collector_config management command configures the VMware data collector. It is intended for appliances running in collector mode.

### Positional arguments

| Argument | Description | Required |
| --- | --- | --- |
| sub command | Available Sub Commands:<br>csmp<br>vmware<br>recurring-job<br>proxy<br>openstack<br>setup<br>cb-platform-client | Y |
| operation | Sub-command operation: specific to each sub-command, described in the sections below. | Y |

### Sub-commands

The collector_config command contains the following sub-commands.

Most operations act on one configuration and take `--config-id` to identify it. Get the ID from that sub-command's **list** operation.

#### csmp

The csmp command is used to manage the CSMP VMware Data Collector configuration.

##### create {#create-csmp}

Create the **CSMP Data Collector** Connection Info that the collector uploads to. Only one can exist.

| Argument | Description | Required |
| --- | --- | --- |
| --domain-url | CSMP domain URL | Y |
| --api-key | CSMP API Key: if not specified, the user will be prompted for it. | N |
| --skip-verify | Include this to skip CSMP connection verification. | N |

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config csmp create --domain-url https://mycsmp.cloudbolt.app --api-key yourapikey
```

##### list {#list-csmp}

List VMware Data Collector configurations; requires no command-line arguments.

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config csmp list
```

Sample Output

```text
+--------------+------------------------------+
|      ID      |       CSMP Domain URL        |
+--------------+------------------------------+
| CON-xfxnxbat | https://mycsmp.cloudbolt.app |
+--------------+------------------------------+
```

##### update-apikey

Update a CSMP API Key.

| Argument | Description | Required |
| --- | --- | --- |
| --config-id | The ID of the config you want to change the API Key for. This can be obtained by using the list operation for this command. | Y |
| --api-key | CSMP API Key: if not specified, the user will be prompted for it. | N |

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config csmp update-apikey --config-id CON-xfxnxbat --api-key yournewapikey
```

##### remove {#remove-csmp}

Remove a VMware Data Collector configuration.

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config csmp remove --config-id CON-xfxnxbat
```

#### recurring-job

The recurring-job command is used to get information about "CSMP VMware Data Collector" jobs

##### list {#list-recurring-job}

List all VMware Data Collector recurring jobs.

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config recurring-job list
```

##### create {#create-recurring-job}

Create a VMware Data Collector recurring job for a VMware configuration. **vmware create** already creates one, so use this only to replace a job that was removed.

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config recurring-job create --config-id RH-ly17mzoa
```

##### update

Update information about recurring jobs, such as dry_run, interval_time, schedule, and max_retries. Provide `--config-id` and at least one of the other options.

| Argument | Description | Required |
| --- | --- | --- |
| --config-id | The ID of the Recurring Job config that the job run belongs to. This can be obtained by using the list operation for this command. | Y |
| --max-retries | Specify the maximum number of retries for the recurring job | N |
| --interval-time | Specify the retry interval (in seconds). | N |
| --schedule | Specify the schedule for the recurring job (for example, `0 1 * * *`) | N |
| --dry-run | Whether to run the recurring job in dry-run mode, without executing any actions: `yes` or `no`. | N |

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config recurring-job update --config-id RJB-702701o7 --max-retries 2 --interval-time 180
```

##### enable {#enable-recurring-job}

Enable a VMware Data Collector Recurring Job.

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config recurring-job enable --config-id RJB-702701o7
```

##### disable {#disable-recurring-job}

Disable a VMware Data Collector Recurring Job.

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config recurring-job disable --config-id RJB-702701o7
```

##### remove {#remove-recurring-job}

Remove a VMware Data Collector Recurring Job.

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config recurring-job remove --config-id RJB-702701o7
```

##### run-now {#run-now-recurring-job}

Manually initiate a VMware Data Collector Job

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config recurring-job run-now  --config-id RJB-702701o7
```

##### list-jobs

List the summary of the last 50 jobs (or `--display-limit`) for a CSMP VMware Data Collector, including start and end dates, and status.

| Argument | Description | Required |
| --- | --- | --- |
| --config-id | The ID of the Recurring Job config you want to list jobs for. This can be obtained by using the list operation for this command. | Y |
| --job-status | Optional: list jobs by job status. | N |
| --start-date | Optional: list jobs by start date in DD-Mon-YYYY format. | N |
| --end-date | Optional: list jobs by end date in DD-Mon-YYYY format. | N |
| --display-limit | Optional: specify the number of jobs to list; defaults to the 50 most recent. | N |

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config recurring-job list-jobs --config-id RJB-702701o7
```

Sample Output

```text
+--------------+----------------------+----------------------+---------+-------------------------------+
|      ID      |      Start Date      |       End Date       |  Status |            Log File           |
+--------------+----------------------+----------------------+---------+-------------------------------+
| JOB-xb855j0c | 26-Mar-2024 18:43:07 | 26-Mar-2024 18:55:28 | SUCCESS | /var/log/cloudbolt/jobs/3.log |
| JOB-judyl6e4 | 26-Mar-2024 18:40:09 | 26-Mar-2024 18:40:11 | FAILURE | /var/log/cloudbolt/jobs/4.log |
| JOB-mqk7p29f | 25-Mar-2024 21:01:25 | 25-Mar-2024 21:03:01 | SUCCESS | /var/log/cloudbolt/jobs/5.log |
+--------------+----------------------+----------------------+---------+-------------------------------+
```

##### get-job

Get information about job runs, including start and end dates, and status.

| Argument | Description | Required |
| --- | --- | --- |
| --config-id | The ID of the Recurring Job config you want to list jobs for. This can be obtained by using the list operation for this command. | Y |
| --job-id | The ID of the job you want to view. This can be obtained by using the list operation. | Y |

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config recurring-job get-job --config-id RJB-702701o7 --job-id JOB-21nnojnu
```

#### vmware

The VMware command is used to manage one or more VMware Resource configurations.

##### create {#create-vmware}

Create VMware configurations.

| Argument | Description | Required |
| --- | --- | --- |
| --host | vCenter Hostname or IP address. | Y |
| --port | vCenter Port; defaults to 443. | N |
| --username | vCenter Username. | Y |
| --skip-ssl | Include this to skip SSL certificate verification when making vCenter API calls; defaults to False when not included on the command line. | N |
| --password | vCenter Password: if not specified, the user will be prompted for it. | N |
| --skip-verify | Include this to skip VMware connection verification. | N |

##### list {#list-vmware}

List VMware resource configurations; requires no command-line arguments

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config vmware list
```

##### update-password

Update your vCenter password

| Argument | Description | Required |
| --- | --- | --- |
| --config-id | The ID of the config you want to change the password for. This can be obtained by using the list operation for this command. | Y |
| --password | vCenter Password; if not specified, the user will be prompted for it. | N |

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config vmware update-password --config-id RH-mzk1bw32 --password yournewvcenterpassword
```

##### remove {#remove-vmware}

Remove VMware Resource configuration.

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config vmware remove --config-id RH-mzk1bw32
```

#### proxy

Manage proxy server settings.

##### create {#create-proxy}

Configure a proxy server.

| Argument | Description | Required |
| --- | --- | --- |
| --host | The proxy server IP or hostname | Y |
| --protocol | The proxy server protocol: if not specified, a proxy is set up for HTTP and HTTPS | N |
| --port | The proxy server protocol port; defaults to 443. | N |
| --username | The proxy server username | N |
| --password | The proxy server password | N |
| --bypass-proxy-domains | A comma-separated list of the domains for which CMP should not attempt to use a proxy server to route the connections. These domain names are regular expressions and could also be represented as IPv4 IP addresses. For example, `'.*\.internal\.loc,192\.168\..*,my\.domain\.com'`. | N |

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config proxy create --host myproxy.mydomain.net --port 3128 --protocol http --bypass-proxy-domains ".*\.mydomain\.net,.*\.mydomain2\.net"
```

##### list {#list-proxy}

List proxy servers

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config proxy list
```

##### update-proxy-bypass

Update proxy bypass rules.

| Argument | Description | Required |
| --- | --- | --- |
| --bypass-proxy-domains | A comma-separated list of the domains for which CMP should not attempt to use a proxy server to route the connections. These domain names are regular expressions and could also be represented as IPv4 IP addresses. For example, `'.*\.internal\.loc,192\.168\..*,my\.domain\.com'`. If not specified, all bypass rules are removed. | N |

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config proxy update-proxy-bypass --bypass-proxy-domains ".*\.mydomain\.net,.*\.mydomain2\.net"
```

##### remove {#remove-proxy}

Remove proxy server configuration.

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config proxy remove
```

##### verify-domain

Verify whether a request for a domain will be routed through the proxy or bypassed.

| Argument | Description | Required |
| --- | --- | --- |
| --request-domain | The request domain to verify whether requests for this domain will be routed through the proxy. | Y |

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config proxy verify-domain --request-domain google.com
```

## Recurring job: Agent - Resend Processed Queue Files

The **Agent - Resend Processed Queue Files** recurring job resends processed ZIP files to the CSMP endpoint if the CSMP host is unreachable. It is created disabled; enable it with `collector_config recurring-job enable`. Additionally, the management commands listed below can be used to manually run the Agent - Resend Processed Queue Files job.

### run-now {#run-now-agent}

Manually run the **Agent - Resend Processed Queue Files** job.

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config recurring-job run-now --config-id RJB-702701o7
```

Output

```text
Successfully initiated Agent - Resend Processed Queue Files job, Job ID: JOB-12wb0q4o
```

### enable {#enable-agent}

Enable the **Agent - Resend Processed Queue Files** recurring job.

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config recurring-job enable --config-id RJB-702701o7
```

### disable {#disable-agent}

Disable the **Agent - Resend Processed Queue Files** recurring job.

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config recurring-job disable --config-id RJB-702701o7
```

### remove {#remove-agent}

Remove the **Agent - Resend Processed Queue Files** recurring job.

Sample Usage

```text
/opt/cloudbolt/manage.py collector_config recurring-job remove --config-id RJB-702701o7
```

### If the job is missing

If the **Agent - Resend Processed Queue Files** job is not listed when you run this command:

```text
/opt/cloudbolt/manage.py collector_config recurring-job list
```

Create it with this command:

```text
/opt/cloudbolt/manage.py collector_config setup initialize
```