Open topic with navigation

# Running the ArcSight Platform OMT Upgrade (AWS)

The 23.3 release requires upgrading the underlying infrastructure of the ArcSight Platform to version 2023.05 (the new OPTIC Management Toolkit, abbreviated OMT). This process can take a significant amount of time, depending on the number of master and worker nodes that need to be updated, so please select the most convenient (less busy) time to perform the upgrade.

All commands should be performed from the bastion host.
Ensure the latest version of the AWS CLI is installed on the bastion. Refer to Installing or updating the latest version of the AWS CLI for instructions.

If the AWS CLI version needs to be updated, execute the following command to update the configuration after the AWS CLI update is finished:

    aws eks update-kubeconfig --name <eks_cluster_name> --region <your_region>

1. Log in to the secure network location where you stored the ArcSight Platform Cloud Installers. Move the downloaded files into the bastion, and perform the commands listed in this procedure from the bastion.

2. Unzip the `arcsight-platform-cloud-installer-<VERSION>.zip` OMT installer file with this command:

       cd /tmp
       unzip arcsight-platform-cloud-installer-<VERSION>.zip

3. On the bastion, verify that all pods in core namespaces are in status Running or Completed by running this command:

       kubectl get pods -n core

   Output example:

       cdf-apiserver-7965dcf689-4qvkx                         2/2     Running     0          145m

       fluentd-7q4dw                                          2/2     Running     0          136m

       fluentd-kkf2p                                          2/2     Running     0          136m

       fluentd-mwqh8                                          2/2     Running     0          136m

       idm-77b4f9fbfb-cfwkg                                   2/2     Running     0          136m

       idm-77b4f9fbfb-g5pcb                                   2/2     Running     0          136m

       itom-cdf-deployer-2020.05-2.2-2.3-3.1-tncp8            0/1     Completed   0          137m

       itom-cdf-deployer-xg6cw                                0/1     Completed   0          147m

       itom-cdf-ingress-frontend-56c9987b7-bvrsn              2/2     Running     0          145m

       itom-cdf-ingress-frontend-56c9987b7-n8tbc              2/2     Running     0          145m

       itom-logrotate-deployment-6cf9546f8b-rbcvs             1/1     Running     0          136m

       itom-postgresql-default-77479dfbff-t87tv               2/2     Running     0          137m

       itom-vault-6f558dc6cc-bz52l                            1/1     Running     0          146m

       kubernetes-vault-67f8698568-csd54                      1/1     Running     0          145m

       mng-portal-7cfc584db5-hcmjf                            2/2     Running     0          133m

       nginx-ingress-controller-6f6d4c95b9-7fhbs              2/2     Running     0          133m

       nginx-ingress-controller-6f6d4c95b9-nv2zw              2/2     Running     0          133m

       suite-conf-pod-arcsight-installer-86c9687b69-kctjz     2/2     Running     0          132m

       suite-db-68bfc4fbd5-v6nvm                              2/2     Running     0          145m

       suite-installer-frontend-6f49f88797-msb7j              2/2     Running     0          145m

4. Also on the bastion, check that all nodes are in Ready state by running this command:

       kubectl get nodes

5. Switch to the AWS scripts installer directory:

       cd arcsight-platform-cloud-installer-<VERSION>.zip

6. Unzip `aws-scripts.zip`:

       unzip aws-scripts.zip

7. Switch to the scripts directory:

       cd aws-scripts/scripts

8. Run the following command with your credentials from the ECR:

       ./upload_images_to_ECR -o $(kubectl get cm -n core base-configmap --output=jsonpath={.data.REGISTRY_ORGNAME}) -F arcsight-platform-cloud-installer-<VERSION>.zip/cdf-byok-images.tar -c 4

   Adjust the value of the `-c` parameter (4 in the instruction above) to up to half your CPU cores in order to increase the speed of the upload (default value is 8).

9. Change directory to `/tmp` directory:

       cd /tmp/

10. Unzip cdf-deployer.zip:

        unzip cdf-deployer.zip

11. Change to the cdf-deployer directory:

        cd cdf-deployer/

12. Run the upgrade script with the following command, and follow the prompts:

        ./upgrade.sh -u

    Output example:

        ***********************************************************************************

        WARNING: This step is used to upgrade CDF components to 2022.05 release.

        The upgrade process is irreversible. You can NOT roll back.

        Make sure that all nodes in your cluster are in Ready status.

        Make sure that all Pods and Services are Running.

        ***********************************************************************************

        Please confirm to continue (Y/N): y

        ** Pre-checking before upgrade ...

        [1]  Checking K8S version ...

             Passed

        [2]  Checking HA Default DataBase ...

             Passed

        [3]  Checking Prometheus ...

             Passed

        [4]  Checking SAN in certificates ...

             Passed

        [5]  Checking connectivity to registry ...

             Passed

        [6]  Checking OS version for bastion node ...

             Passed

        ** Common upgrade check ...

        [1]  Checking cluster endpoints status ...

             Passed

        [2]  Checking chart releases status ...

             Passed

        ** Prerequisite tasks for Apphub components upgrade ... (Step 1/4)

             Setting BYOK environment files ...

             Copying itom-cdf.sh to HOME folder ...

             Copying Scripts ...

        ** Updating Kubernetes RBAC ... (Step 2/4)

             RBAC update successfully.

             Updating yaml-templates configmap ...

        ** Start to upgrade Apphub add-on charts ... (Step 3/4)

             Apphub add-on charts have been successfully updated.

        ** Start to upgrade Apphub chart ... (Step 4/4)

             Apphub chart has been successfully updated.

        Successfully completed Apphub components upgrade process.

    The command `upgrade.sh -u` may return an error message on Apphub. If this occurs, run the following command:

    `helm rollback apphub 1 -n core`

    After the command has executed, re-run the upgrade.

13. When the upgrade completes, check that all pods are Running or Completed with this command:

        kubectl get pods -A

Once the OMT upgrade is complete, the new platform version can be checked by clicking the ? icon in the upper right corner of the OMT UI

Next Step: Configuring Controller Services