Knowledge Discovery 25.1
Getting Started Guide

# Linux System Recommendations

This section describes some Linux operating system settings that you might want to modify to improve Knowledge Discovery performance and function. You might need additional system privileges to make these changes. In all cases, OpenText recommends that you test on your own systems to find the optimal settings for your setup.

NOTE: For many of the settings listed in this section, values and example commands are given for Red Hat Enterprise Linux (RHEL). You might need to check the default values and suggested commands for other Linux distributions.

## Open File Descriptors

The Knowledge Discovery Content component can use a large number of files for certain tasks, such as indexing, particularly for large indexes. On Linux systems, the number of file descriptors is limited to prevent them from using too much memory. The memory usage of a single file descriptor is low, so it is usually safe to set the limit to a high value. OpenText recommends that you increase the limit value to at least `65536` for the user that the Content component runs as.

You can find an estimate of the number of files that a process (with a specified process ID) is using by using the following command:

    ls -l /proc/ProcessID/fd | wc -l

You can find the current limit for the number of files by using the following commands:

    Ulimit –Sn
    Ulimit -Hn

There are two limits to the number of files that processes can use:

- The soft limit is the default limit at the start of a user session, and you can use `ulimit` commands to increase it (up to the hard limit).
- The hard limit is the absolute limit for your system.

To modify the individual process file limit for the user that the Content component runs as (`UserName`), open the `/etc/security/limits.conf` configuration file, and add or modify the following lines to modify the soft and hard file limits (where `NewValue` is the limit that you want to set):

    UserName hard nofile NewValue
    UserName soft nofile NewValue
    UserName hard nproc NewValue
    UserName soft nproc NewValue

TIP: On Ubuntu, when you use the start-stop-daemon, you might need to run the `ulimit` command in the init script to update the hard and soft limits when you start the service.

If you start Knowledge Discovery component executable files directly (rather than using the init script) by using SSH and `su`, you might also need to modify the `/etc/pam.d/su` file to uncomment the following line:

    # session    required   pam_limits.so

## File Handle Limits in Services

On systemd init systems, services do not inherit file handle limits from the system limits or user settings. In some cases, this behavior might mean that a component fails to operate when running as a service, because it runs out of file handles.

You can modify the file handle limits for the service by adding the `LimitNOFILE` parameter to the service file, or by creating an `override.conf` file for the service.

For more information, see Install a Knowledge Discovery Component as a Service on Linux.

## Transparent Huge Pages

Transparent Huge pages (THP) is the use of an abstraction layer that automates the creation, management and use of Huge Pages for memory management. In RHEL 6 and later, this feature is turned on by default, but Red Hat does not recommend the use of THP for database workloads. For Knowledge Discovery, turning this setting off can improve the performance for many query operations.

You can change this setting in a terminal by using the following commands:

    echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
    echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag

TIP: On Ubuntu, you can use the following commands:

    echo never > /sys/kernel/mm/transparent_hugepage/enabled
    echo never > /sys/kernel/mm/transparent_hugepage/defrag

You can also turn this setting off at boot time, by adding the following text to the bottom of the `/etc/rc.local` file:

    #disable THP at boot time
    if test -f /sys/kernel/mm/redhat_transparent_hugepage/enabled; then
    echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
    fi
    if test -f /sys/kernel/mm/redhat_transparent_hugepage/defrag; then
    echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag
    fi

TIP: On Ubuntu, the `/etc/rc.local` ends with the line `exit 0`, and you must make any changes to the file above that line.

## Tune Virtual Memory

OpenText recommends tuning the virtual memory usage for Linux systems for database workloads to improve Knowledge Discovery performance. In particular, you might want to modify the following settings:

- `vm.swappiness`. This setting controls how much the system favors swapping out runtime memory. The default value on Linux systems is `60`, but for database workloads, the recommended value is `10`. OpenText recommends that you set this value to `10` or lower. A value of `0` or `1` configures the system to only use the minimum amount of swapping, but does not turn it off.
- `vm.dirty_ratio`. This setting controls the maximum percentage of memory that can contain pages that have not yet been written to disk. The default value is `20`, but for database workloads the recommended value is `15`.
- `vm.dirty_background_ratio`. This setting controls the percentage of memory that can contain pages that have not yet been written to disk before the system starts to write the data in the background. The default is `10`, but for database workloads the recommended value is `3`.

You can check the current values for these settings by running the following command in the terminal:

    # sysctl –a | grep "vm.Parameter"

To change these settings permanently, update the `/etc/sysctl.conf` configuration file, to add or modify the following lines:

    vm.swappiness = 10
    vm.dirty_ratio = 15
    vm.dirty_background_ratio = 3

## Memory Map Counts

The Content component can use a large number of memory map areas for certain tasks, such as indexing, particularly for large indexes. On Linux systems, the number of memory mapped areas that a process can use is limited, to a value controlled by the `max_map_count` kernel parameter. To prevent Knowledge Discovery from reaching this limit, OpenText recommends that you increase the value of the `max_map_count` parameter to the amount of memory on the system in KB /16, but no less than 65536.

NOTE: This recommendation applies for a single Content component running on the host machine. If you have multiple Content components running on the same host, you might need to limit this value further.

You can find an estimate of the number of memory mapped areas that a process (with a specified process ID) is using by using the following command:

    cat /proc/processID/maps | wc -l

You can find the current value of `max_map_count` by using the following command:

    sysctl vm.max_map_count

You can change the value of `max_map_count` by using the following command:

    sysctl vm.max_map_count=NewValue

## Configure the I/O Scheduler

The Linux kernel can use several different I/O schedulers to prioritize disk input and output. By default, many Linux distributions use the Completely Fair Queuing (CFQ) scheme for I/O scheduling, which gives input and output requests equal priority. For Knowledge Discovery systems, OpenText recommends that you set I/O scheduling to either `deadline` or `noop`:

- The Deadline scheduler gives priority to read requests over write requests. It also imposes a deadline on all requests. After reaching the deadline, these requests gain priority over all other requests. This scheduling method helps prevent processes from becoming starved for I/O access. The Deadline scheduler is recommended for physical media drives (HDD), because it attempts to group requests for adjacent sectors on disk, reducing the amount of time that the disk spends seeking.
- The NOOP scheduler uses a simple first-in first-out approach, placing all input and output requests into a single queue. This scheduler is recommended for solid state drives (SSDs). Because SSDs do not have a physical read head, there is no performance penalty for accessing non-adjacent sectors.

OpenText recommends that you always use one of these schedulers, rather than the CFQ scheduler. However, the exact method to use depends on your environment and usage, so you might need to test both of these methods on your system to see which provides the best performance.

You can check whether your disks are SSD or HDD by using the following command:

    for i in /sys/block/sd*; do cat $i/queue/rotational; done

This command returns `1` for HDD, and `0` for SSD.

You can find the scheduler for your mounted devices by using the following command:

    for i in /sys/block/sd*; do cat $i/queue/scheduler; done

NOTE: On RHEL 6 and above, for multipath devices, you must also change the `dm-*` block device.

This command returns a result of the following type, with the current option surrounded with square brackets:

    noop deadline [cfq]

You can change the scheduler by using a command of the following type:

    for i in /sys/block/sd*; do echo "noop" >  $i/queue/scheduler; done

This example changes the system to use the NOOP scheduler.

Copyright 2024 Open Text
Version 25.1 | Last updated January 2025

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: Knowledge Discovery 25.1

Topic Title: Linux System Recommendations