Knowledge Discovery 25.1
Knowledge Discovery Expert

# The Index Process in a Distributed System

When you index data using a DIH, there are more stages in the index process than when indexing directly into the Content component. This is because, by default, DIH continues to poll its child server after it forwards an index action, and does not mark the job as complete until all child servers have finished processing.

TIP: You can turn off polling by setting the `Polling` configuration parameter to `False` in the DIH configuration file.

The following diagram describes the indexing process for the DIH, from when it receives the index action to when all child servers finish processing. The colored boxes correspond to the status that the `IndexerGetStatus` action returns for that stage of the process:

For more information about `IndexerGetStatus` and the status messages that it returns, refer to the Content Component Help.

Index Process for a Distributed System

Incoming index actions have a status of "Initializing" until DIH processes them. In mirror mode, all DIH does is add the index action to the outgoing queues for all children. In other modes, the DIH might also modify the index action or the data in the index file, or define which child servers to send the index action to.

You can view the index queue and associated information on the Index Queue tab of the Status page in Knowledge Discovery Admin.

After the DIH processes an index action, it maintains a record of the index status of each child server for the job. For a child server that does not need to receive the index action, the DIH immediately assigns a child status of "Finished". It assigns other child servers a status of "To be sent to the DRE" until it has forwarded the index action and received an Index ID from the child server, which it uses to poll the index action status in the child server. It polls the status and updates its own record until the child server returns a permanent status; that is, "Finished" or one of the failure codes.

TIP: You can check the individual status of all DIH child servers by sending an `IndexerGetStatus` action with the `ChildDetails` parameter set to `True`. In addition, you can use the `Child` parameter to restrict the list of child server statuses to a subset of child servers.

TIP: You can force the DIH to poll its children, or to return its cached values for child statuses, by using the `PollChildren` parameter. This parameter is available as a configuration option, which you can override with the action parameter for an individual `IndexerGetStatus` action.

NOTE: The child server of a DIH can be a DIH, so the child statuses are not restricted to the set that the Content component returns.

The DIH reports an aggregate status of the statuses reported by its children:

- If any child server has a status of "Processing", DIH returns this status.

- If any child server has another temporary status, DIH returns this status.

- If all child servers have the same permanent status ("Finished" or a failure code), DIH returns this status.

- If child servers have different permanent statuses, DIH returns "Child engines returned differing messages".

It is possible that when DIH polls, the child server does not recognize the child index ID that DIH requests. In this case, the DIH assigns the child server a status of "The Index ID specified is invalid."

This might occur if the child server has processed a `DREINITIAL` index action, or if the DIH has a long polling interval and the index action has been finished and removed from the child server index queue history between poll requests.

The DIH does not process the `Pause` or `Restart` options for the `IndexerGetStatus` action `IndexAction` parameter, and it does not forward these operations to its child servers. When you use the `Cancel` option:

- if the DIH is processing the job, it stops processing.

- if the DIH has finished processing but has not sent the job to the child servers, it does not send it.

- if the job is being processed in child servers and it has a temporary status, DIH stops polling the job. DIH does not forward the `Cancel` request to the child servers.

Copyright 2024 Open Text
Version 25.1 | Last updated January 2025

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: Knowledge Discovery 25.1

Topic Title: The Index Process in a Distributed System