When you index data using a DIH, there are more stages in the index process than when indexing directly into the IDOL Server Content component. This is because, by default, DIH continues to poll its child server after it forwards an index action, and does not mark the job as complete until all child servers have finished processing.
You can turn off polling by setting the Polling configuration parameter to False in the DIH configuration file.
The following diagram describes the indexing process for the DIH, from when it receives the index action to when all child servers finish processing. The colored boxes correspond to the status that the IndexerGetStatus action returns for that stage of the process:
For more information about IndexerGetStatus and the status messages that it returns, refer to the IDOL Server Reference.
Index Process for a Distributed System
Incoming index actions have a status of "Initializing" until DIH processes them. In mirror mode, all DIH does is add the index action to the outgoing queues for all children. In other modes, the DIH might also modify the index action or the data in the index file, or define which child servers to send the index action to.
You can view the index queue and associated information on the Index Queue tab of the Status page in IDOL Admin.
After the DIH processes an index action, it maintains a record of the index status of each child server for the job. For a child server that does not need to receive the index action, the DIH immediately assigns a child status of "Finished". It assigns other child servers a status of "To be sent to the DRE" until it has forwarded the index action and received an Index ID from the child server, which it uses to poll the index action status in the child server. It polls the status and updates its own record until the child server returns a permanent status; that is, "Finished" or one of the failure codes.
You can check the individual status of all DIH child servers by sending an IndexerGetStatus action with the ChildDetails parameter set to True. In addition, you can use the Child parameter to restrict the list of child server statuses to a subset of child servers.
You can force the DIH to poll its children, or to return its cached values for child statuses, by using the PollChildren parameter. This parameter is available as a configuration option, which you can override with the action parameter for an individual IndexerGetStatus action.
The child server of a DIH can be a DIH, so the child statuses are not restricted to the set that the Content component returns.
The DIH reports an aggregate status of the statuses reported by its children:
If any child server has a status of "Processing", DIH returns this status.
If any child server has another temporary status, DIH returns this status.
If all child servers have the same permanent status ("Finished" or a failure code), DIH returns this status.
If child servers have different permanent statuses, DIH returns "Child engines returned differing messages".
It is possible that when DIH polls, the child server does not recognize the child index ID that DIH requests. In this case, the DIH assigns the child server a status of "The Index ID specified is invalid."
This might occur if the child server has processed a DREINITIAL index action, or if the DIH has a long polling interval and the index action has been finished and removed from the child server index queue history between poll requests.
The DIH does not process the Pause or Restart options for the IndexerGetStatus action IndexAction parameter, and it does not forward these operations to its child servers. When you use the Cancel option:
if the DIH is processing the job, it stops processing.
if the DIH has finished processing but has not sent the job to the child servers, it does not send it.
if the job is being processed in child servers and it has a temporary status, DIH stops polling the job. DIH does not forward the Cancel request to the child servers.
|
|