The most common causes of index corruption are:
process termination (when a process is forcibly stopped or stops unexpectedly, or when the hardware is switched off)
storage failures (connection to network storage broken, or disk failure)
RAM problems (typically rare)
You can normally tell whether the IDOL process was terminated abnormally by inspecting the IDOL Server logs and seeing if the Version
message is preceded by a clean stop message. If there is no clean stop message, it suggests the previous run was somehow interrupted. System logs also often give an idea of whether a process was terminated abnormally. Storage or RAM failures are typically more difficult to detect and might require additional system tools.
IDOL Server usually detects if it was previously interrupted, and on startup it attempts to validate the internal indexes. During this startup validation the process is unavailable for querying or indexing. You can override this behavior by setting the ValidateOnInterruption
to False
. IDOL Server also typically rolls back any index actions that were not committed, and automatically reattempts them.
You can run validation at any time when the server is running by using the DREVALIDATE
index action, or by using the Validate feature on the Service Control tab of the Console page in the Control section of IDOL Admin.
No indexing occurs during the validation process, but the system is still available for queries. Validation is normally an I/O intensive process so it might reduce query performance.
After validation is complete and the server is available for query, IDOL Server displays the result of the validation in the GetStatus
response. The GetStatus
response always includes information from the last available validation record. If any index has failed validation then the IDOL Server logs provide more details about the errors detected.
In some cases it is desirable to prevent the system from running with a corrupted index or to stop further validation as soon as a single error has been found. You can control this behavior by using the ValidateOptions
configuration parameter.
The diskindex
and nodetable
indexes are the most fundamental structures in IDOL, and failures in these structures typically means that a server might need to be restored from backup or from source data. In some cases the errors might be isolated to document ranges, in which case it might be possible to just delete those documents and reindex them. You can regenerate the other data structures (numeric
, match
, parametric
and so on) using the Regenerate*Index
configuration parameters. IDOL Server rebuilds all these structures using data held in the nodetable.
If either of the diskindex
or nodetable
indexes are damaged, you can use the following approaches to recover the data (listed in order of speed from fastest to slowest:
Restore from a DREBACKUP
(possibly in combination with partial application of some of the following items).
Replay archived index actions (to do this, you must configure an ArchivePath
to store index actions).
Reindex any stored IDX or XML files generated by the connectors.
DREEXPORT
from valid nodetable
data and reindex into a fresh server to rebuild the diskindex
(and all other indexes).
Rerun the import process (effectively restarting the entire import and index process).
|