Open topic with navigation

# Installing the Database in AWS

This section provides information about installing the ArcSight Database in Amazon Web Services Deployment (AWS).

- Understanding Methods for Connecting to AWS S3 Buckets

- Launching Database Instances in AWS

- Creating the Swap File

- Formatting Devices for the Installation

- Persisting Operating System Settings

- Enabling Root Login for AWS Passwordless Communication

- Installing the Database in AWS

## Understanding Methods for Connecting to AWS S3 Buckets

The database uses a single communal storage location for all data and for the catalog (metadata). Communal storage is the database's centralized storage location, shared among the database nodes. This mode supports communal storage in Amazon S3, which must be set up by your cloud administrator before you can install the database.

Other prerequisites or considerations for the S3 bucket:

- Set up default encryption for the S3 bucket in advance. For information about enabling S3 bucket encryption, see AWS documentation, Enabling Amazon S3 default bucket encryption.

- If you require a folder under your S3 bucket, it must be created in the communal storage procedure. Folders pre-created under the bucket via the AWS console are not supported for a new installation of the database.

  For more information, see Configuring and Installing the Database Server

- The database supports connecting to AWS S3 buckets using IAM roles. IAM roles are the default access control method for AWS resources. The database uses this method if you do not configure the legacy access control session parameters.

  To use an IAM role, the bucket must be in the same region as the database cluster and the role needs to be set with the proper permissions for reading and writing to the S3 bucket.

  For more information about IAM roles, see AWS documentation, IAM Roles for Amazon EC2 and Creating a role to delegate permissions to an AWS service.

  The example policy below shows the permissions needed for the IAM policy role:

       {
      "Sid": "s3CommunalLocationAccess",
      "Action": [
      "s3:ListBucket",
      "s3:GetObject",
      "s3:DeleteObject",
      "s3:PutObject",
      "s3:*MultipartUpload*"
      ],
      "Resource": [
      "arn:aws:s3:::<db-communal-storage-bucket-name>/*",
      "arn:aws:s3:::<db-communal-storage-bucket-name>"
      ]
      }

  In the example above you would replace the <db-communal-storage-bucket-name> with the name of the S3 bucket you created for the database communal storage. For example, `222222222-ap-southeast-1-arcsight-db`

## Launching Database Instances in AWS

The ArcSight Database compute operations are performed on a cluster of AWS EC2 (Elastic Compute Cloud) nodes. These nodes must be deployed in AWS before the database can be installed on them.

1. Sign in to the Amazon EC2 console and select a region.

2. Select Launch Instances.

3. Enter a name for your instance and add required tags.

4. Search for AMIs. The database supports Red Hat 8.4 AMIs. For information about finding an AMI, see AWS documentation, Find a Linux AMI.
   For example. If you are in the US East (N. Virginia) region, you can use the following AMIs:

   - `Red Hat 8.4 (AMI Id: ami-0453b4b64c860454a)`

5. After selecting the AMI, and reviewing the details, select an Instance Type. Choose the AWS EC2 instance type that best matches your requirements.

   We recommend using an m5d instance type, because it has the best balance of cost, performance, and ease of setup for ArcSight workloads. Take into account that m5d is an instance store that provides temporary storage. For example, data files stored on the instance are lost when the instance is stopped. The durable EBS instance is also supported with Elastic Block Store (EBS). This instance type provides durable storage. For example, data files stored on the instance persist after the instance is stopped.
   For more information on which size of m5d (instance store ) and /m4 (EBS) to use for your workload, see the Technical Requirements for ArcSight Platform 23.3.

6. For Key pair, select an existing one or create new.

7. Configure your instance with the VPC, private subnets and security groups.

8. In the Configure storage section, change the size for the Root volume type to 30 GiB, the minimum size for root, or higher.

   If you use the EBS Instance type, m4, add an EBS volume that follows the System Performance Benchmarks for Sizing and Tuning in the Technical Requirements for ArcSight Platform 23.3.
9. In the Advanced details section, go to the IAM instance profile (IAM role) and select the IAM profile you configured to access the S3 bucket for database communal storage. For more information, see Understanding Methods for Connecting to AWS S3 Buckets.
10. In the Summary section, configure the number of AWS EC2 instances to deploy. A minimum of three is required for a highly available database cluster, but more might be necessary to handle your workload. For more information on how many instances you need, see the Technical Requirements for ArcSight Platform 23.3.

11. Configure the number of AWS EC2 instances to deploy. A minimum of three is needed for a highly available database cluster. For more information, on how many instances you need, see the Technical Requirements for ArcSight Platform 23.3.

12. Click Launch Instances.

13. Navigate to EC2, and verify your instances are available.

## Creating the Swap File

A partition with swap space configured is required to enable virtual memory access. The database requires a minimum of 2 GB of swap space on all database nodes. Perform the following procedure on all database nodes to setup the swap space.

1. From the bastion, login to the database node using SSH and the key pair selected during instance launching. For example:

       ssh -i <FilePath to the ec2 keypair/IdentityFile> root@<node1..n>

2. Create the file that will be used for swap:

       sudo dd if=/dev/zero of=/swapfile bs=1024 count=2621440

   The count 261440 is the minimum value required for installation, but it is also an example. You can set the value higher.

3. Enable read and write to the swap file for the root user (only):

       sudo chmod 600 /swapfile

4. Set up the file as a Linux swap area:

       sudo mkswap /swapfile

5. Enable the swap with the following command:

       sudo swapon /swapfile

6. Enable the swap permanently as root user by editing the`/etc/fstab` file and appending the following line:

       echo "/swapfile swap swap defaults 0 0" | sudo tee -a /etc/fstab

7. Verify that swap is enabled:

       sudo swapon --show

       Output example:

       NAME      TYPE SIZE USED PRIO
       /swapfile file 2.5G   0B   -2

## Formatting Devices for the Installation

The database requires a storage device formatted in ext4 for all nodes. Perform the following procedure on all database nodes to setup ext4 in devices.

The steps below are required to be performed on all AWS EC2 instances that are created. For more information about creating instances, see Launching Database Instances in AWS

Format the device type applicable to the AWS EC2 type you are using in the instance, either an instance Store type or EBS type:

- Formatting Instance Store Type Devices

- Formatting EBS Type Devices

### Formatting Instance Store Type Devices

The steps below are required to be performed on all AWS EC2 instances that are created.

ONLY perform this procedure if you are using Instance Store type devices.

1. From the bastion, login to the database node using SSH and the key pair selected during launching.
   For example:

       ssh -i <FilePath to the ec2 keypair/IdentityFile> root@<node1..n>

2. Install the `nvme` tool:

       sudo dnf install -y nvme-cli

3. Create an `/opt mount` point. If`/opt` already exists, skip this step.

       sudo mkdir /opt/vertica

4. Use the `nvme` tool to locate the `nvme` device and format it:

       EPHEMERAL_DISK=$(sudo nvme list | grep 'Amazon EC2 NVMe Instance Storage' | awk 'NR==1{ print $1 }')

       echo $EPHEMERAL_DISK

       sudo mkfs.ext4 $EPHEMERAL_DISK

       sudo mount -t ext4 $EPHEMERAL_DISK /opt

5. Add the ephemeral disk mount to `/etc/fstab`:

       echo "$EPHEMERAL_DISK /opt ext4 defaults,nofail 0 0" | sudo tee -a /etc/fstab

6. To list the mount points for verification, run the `lsblk` command:

       lsblk

### Formatting EBS Type Devices

The steps below are required to be performed on all AWS EC2 instances that are created. These steps for EBS also assume you are formatting a single device. For more information about creating instances, see Launching Database Instances in AWS.

ONLY perform this procedure if you are using EBS type devices.

1. Run the following command to locate the EBS device attached to the instance, and to select the device that is not mounted:

       lsblk -f

       #Output from lsblk -f
       NAME FSTYPE LABEL UUID MOUNTPOINT
       xvda
       └─xvda1 xfs 35761952-413c-42e8-b047-a5deb7510f29 /
       xvdb

2. Run the following command to format the device to ext4:

       sudo mkfs.ext4 /dev/xvdb

   NOTE: `xvdb` is an example and will be <yourDeviceName> from the output in the previous step.

3. Format the device:

       sudo mount -t ext4 /dev/xvdb /opt

4. Create the mount point. If `/opt` already exists, skip this step.

       sudo mkdir /opt/vertica

5. Modify the `/etc/fstab` file, and add an entry to mount the device on OS boot:

       echo "/dev/xvdb /opt ext4 defaults 0 0" | sudo tee -a /etc/fstab

## Persisting Operating System Settings

The database requires that you manually configure several general operating system settings. Perform the following procedures on all database nodes to setup ext4 in devices.

1. Run this command to set the limit for open files so that it meets database requirements. This will add the parameters to the `/etc/sysctl.conf` file.

       cat << EOF | sudo tee -a  /etc/sysctl.conf
       net.core.somaxconn = 1024
       net.core.wmem_max = 16777216
       net.core.rmem_max = 16777216
       net.core.wmem_default = 262144
       net.core.rmem_default = 262144
       net.core.netdev_max_backlog = 100000
       net.ipv4.tcp_mem = 16777216 16777216 16777216
       net.ipv4.tcp_wmem = 8192 262144 8388608
       net.ipv4.tcp_rmem = 8192 262144 8388608
       net.ipv4.udp_mem = 16777216 16777216 16777216
       net.ipv4.udp_rmem_min = 16384
       net.ipv4.udp_wmem_min = 16384
       vm.swappiness = 0
       EOF

2. Update the`/etc/rc.`local file by running the commands below. This file contains scripts and commands that run each time the system is booted, and the database requires that I/O Scheduling be set to deadline or noop. The command will add the applicable lines to the file, based on the devices type in your AWS instance, either instance Store type or EBS type:

   Instance Store type rc.local settings (only)

       cat << EOF | sudo tee -a /etc/rc.local
       echo deadline > /sys/block/nvme0n1/queue/scheduler
       echo deadline > /sys/block/nvme1n1/queue/scheduler
       echo deadline > /sys/block/nvme2n1/queue/scheduler
       echo deadline > /sys/block/md0p1/queue/scheduler

       /sbin/blockdev --setra 2048 /dev/nvme0n1
       /sbin/blockdev --setra 2048 /dev/nvme1n1
       /sbin/blockdev --setra 2048 /dev/nvme2n1
       /sbin/blockdev --setra 2048 /dev/md0p1

       if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
       echo never > /sys/kernel/mm/transparent_hugepage/enabled
       fi

       if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
       echo never > /sys/kernel/mm/transparent_hugepage/defrag
       fi
       EOF

   Note: Replace the device in the example command above with your own instance device on which the `/opt` is mounted. For example, the following device must be replaced with your own device: `sys/block/nvmen0n1`

   EBS type rc.local settings (only)

       cat << EOF | sudo tee -a /etc/rc.local
       echo deadline > /sys/block/xvda/queue/scheduler
       echo deadline > /sys/block/xvdb/queue/scheduler
       echo deadline > /sys/block/xvdc/queue/scheduler

       /sbin/blockdev --setra 2048 /dev/xvda
       /sbin/blockdev --setra 2048 /dev/xvdb
       /sbin/blockdev --setra 2048 /dev/xvdc

       if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
       echo never > /sys/kernel/mm/transparent_hugepage/enabled
       fi

       if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
       echo never > /sys/kernel/mm/transparent_hugepage/defrag
       fi
       EOF

   Note: Replace the device in the example command above with your own instance device on which the `/opt` is mounted. For example, the following device must be replaced with your own instance device:

   `sys/block/dev/xvdb`

3. Modify the file permissions:

       sudo chmod +x /etc/rc.d/rc.local

4. Run the following commands to disable the system firewall:

       sudo systemctl mask firewalld
       sudo systemctl disable firewalld
       sudo systemctl stop firewalld

   During installation, the database requires that host-based firewalls are disabled on database nodes. After installation, the host-based firewalls can be enabled and the database requires several ports to be open on the local network. We recommend for optimal performance using host-based firewalls between database nodes and a network-based firewall to protect the segment that database cluster is within. However, there is no restriction against using a network-based firewall between database nodes. When using any kind of firewall, ensure that all the database ports are available (see the Technical Requirements for ArcSight Platform 23.3). For more information, see Firewall Considerations in the ArcSight Database Guide.

5. Set SELinux to permissive mode in `/etc/selinux/config`.

       SELINUX=permissive

   For more information, see SELinux Configuration.

6. Run this command to ensure that `rng-tools` packages are installed in all cluster nodes:

       sudo dnf install rng-tools -y

7. Set the UTC time for all cluster nodes:

       sudo timedatectl set-timezone UTC

8. Reboot the system for your changes to take effect.

   This step is required to update the device mounts and other settings.

## Enabling Root Login for AWS Passwordless Communication

All commands require root privileges which can be obtained through the sudo command.

1. Connect to one of your nodes and edit the `/etc/ssh/sshd_config` configuration file.

       sudo vi /etc/ssh/sshd_config

2. Change the following parameter to `yes` if the value is not already set to that.

       PermitRootLogin yes

3. Proceed as follows:

   - If you had to change the `PermitRootLogin` parameter to `yes`, run this command:

         sudo service sshd reload

   - If the `PermitRootLogin` parameter was already set to `yes`, proceed to the next step.

4. Patch the authorized keys file for the root user by copying the RHEL file to the root user. This enables logging in with the same key which is available for the ec2-user (rhel-8.4).

       sudo cp ~<ec2-user>/.ssh/authorized_keys ~root/.ssh/authorized_keys

5. Update the AWS cloud configuration file by editing`/etc/cloud/cloud.cfg` and changing the `disable_root` value to 0.

       sudo vi /etc/cloud/cloud.cfg

6. Repeat all the above steps for all nodes in the cluster.

7. Generate SSH keys for the database nodes:

   1. Log in to one of the database nodes. This becomes the initiator node (node1) for your cluster.

   2. Run the following command as root:

          ssh-keygen -q -t rsa -f ~/.ssh/id_rsa

   3. Run the following command as root, for each of the database nodes (initiator node inclusive):

          ssh-copy-id -f "-o IdentityFile <FilePath to the ec2 keypair>" root@<node1..n>

          Enter : Yes

   4. To verify the passwordless communication, run the following command:

          ssh root@<node1,node2..>

          Output example:

          Activate the web console with: systemctl enable --now cockpit.socket
          Last login: Thu Jan 20 19:26:50 2022 from <node1,node2..>

Next Step: Setting Up and Installing the Database for AWS