Distributed Index Handler 24.1

# Configure Consistent Hashing

The following procedures describe how to configure consistent hashing mode in your DIH and child IDOL servers.

## Configure the DIH for Consistent Hashing Mode

The following procedure describes the configuration changes to the DIH that enable consistent hashing mode. To use consistent hashing, you must configure DIH in DistributeByReference or DistributeByFields mode.

NOTE: After you configure the number of VirtualNodes and VirtualNodes for your system, you cannot change these values. If you want to change these numbers, you must use a clean DIH installation and reindex all your content.

For more information about these parameters, refer to the DIH Reference.

To configure consistent hashing in the DIH

1. Open the DIH configuration file in a text editor.

2. In the `[Server]` section, set the UseConsistentHashing parameter to `True`. In a unified IDOL configuration, set this parameter in the `[DistributionSettings]` configuration section. For example:

       [Server]
       UseConsistentHashing=True

3. Create a `[ConsistentHashing]` section.

4. Set the VirtualNodes parameter to the number of virtual nodes that you want to create. DIH rounds this value up to the nearest power of two (for example, if you set VirtualNodes to `4000`, it creates 4,096 virtual nodes.

   OpenText recommends that you set the value of VirtualNodes to be an order of magnitude higher than the number of child servers that you expect to use.

   The minimum value is the higher number of 33 (that is, 64 nodes), or the number of child servers in your system. If you create fewer virtual nodes than there are child servers, DIH does not start. In general, OpenText recommends that you do not reduce the default value of virtual nodes (4,096).

5. (Optional) Set the Replicas parameter to the number of identical copies of each document that you want to index. When you configure replicas, DIH copies the documents in a particular virtual node to two or more child servers.
6. Save and close the DIH configuration file.

## Configure the Child Servers for Consistent Hashing Mode

The following procedure describes the configuration changes to the child IDOL servers that are required for consistent hashing mode.

To configure consistent hashing for the child servers

1. Open the configuration file for one of your child IDOL or Content servers.

2. In the `[FieldProcessing]` section, add a new field process at the bottom of the list. For example:

       [FieldProcessing]
       0=SetIndexFields
       1=SetReferenceFields
       2=SetSectionBreakFields
       3=SetVNodeReferenceField

   NOTE: You must create a new field process, rather than adding the field to an existing reference process.

3. Create a new configuration section for the field process. Set PropertyFieldCSVs to `*/DREVNODE`, and set Property to the name of a new property. For example:

       [SetVNodeReferenceField]
       Property=VNodeReferenceField
       PropertyFieldCSVs=*/DREVNODE

4. Create a configuration section for the corresponding property. Set the ReferenceType and TrimSpaces properties to `True`. You might also want to use the HiddenType property for this field. For example:

       [VNodeReferenceField]
       ReferenceType=True
       TrimSpaces=True
       HiddenType=True

5. Save and close the configuration file. Restart the child server for your changes to take effect.
6. Repeat Step 1 to Step 5 for each of your child servers.

## Configure the DAH

In consistent hashing mode when you have not configured replicas, you do not need to make any changes to the DAH configuration, except when you add or remove child servers.

When you configure replicas, the DIH distributes identical copies of the virtual nodes between its child servers, ensuring that it does not assign the copies to the same child server. Unlike full server mirroring, the DAH does not know where each of the copies are. In this case, you must:

- configure your DAH in simple combinator mode
- combine query results by reference

For more details about these settings, refer to the DAH Administration Guide and IDOL Server Reference.

In addition, when you send a `GetQueryTagValues` action to the DAH in a system that uses replicas, the document counts for tags include a value for each copy of the document that exists (that is, the action counts each replica as a separate document).

Copyright 2023 Open Text
Version 24.1 | Last updated December 2023

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: IDOL Distributed Index Handler 24.1

Topic Title: Configure Consistent Hashing