Content Manager Enterprise Studio 24.4 Enterprise Studio

# Re-indexing the dataset using IDOL

This function enables you to rebuild the document content indexes, IDOL metadata indexes and Location indexes with the user security settings.

It is not for re-indexing the words in the text fields. For this function, see Index Text.

NOTE: Highly recommended:

- Use a temporary index for re-indexing so that users can continue to work. To do this, create the new index in a new location. When the re-index is complete, copy it to the location of the original index.
- Run the re-indexing process on the Workgroup Server for efficiency and to pick up the defaults

Re-indexing records

You can use this function to select existing Content Manager records with electronic documents to index the document content for the first time or to re-index if they have not been indexed correctly, for example because of a network failure.

You should also use it when an entire Content Manager document content index has been lost or damaged, for an upgrade of a Content Manager 7.2 IDOL index and when the indexing parameters have been changed, for example which Additional Fields to index.

To improve the very first document content indexing process, see Options to improve the initial indexing process below.

Follow the instructions in the Content Manager Help topic IDOL Record Reindex.

Re-indexing user Locations

You can use this function to update the user security data in IDOL by using the CFS connector. This is useful when an administrator has made changes to user security data like Content Manager Access Controls and user permissions.

Follow the instruction in the Content Manager Help topic IDOL Record Reindex.

See also Indexing in Content Manager.

Options to improve the initial indexing process

The longest process in any installation of Content Manager IDOL is the initial indexing of the electronic documents in the dataset. However, there are a number of ways to optimize this process.

1. Exclude unwanted file types from indexing. See Suppressing document content indexing.
2. If there are specific Record Types in your dataset that do not need to have the content of their electronic documents indexed, ensure that you have suppressed document content indexing in the options for the Record Type. In Content Manager, see Content Manager Help about the Record Type Electronic tab.
3. In Content Manager, ensure that you have set the maximum size for archive and standard files in the IDOL Record Reindex dialog box, Select tab.
4. You can spread the indexing load across multiple Workgroup Servers. It is now possible to process document content from more than one Workgroup Server and write to the same content database.
   Redundant Workgroup Servers can be configured to process chunks of data at the same time. The Workgroup Servers would be configured separately and not be part of the same deployment. They would be set up to be identical and configured to use the same IDOL Server. You could use virtual machines for this purpose, with enough disk space to handle the text extraction process and processing of archive files.
   The process uses minimal resources to access the database and document stores and should not have an undue impact on production performance.
   Each Workgroup Server would run as many text extraction processes as it has CPU cores available. Content Manager IDOL includes a deduplication function so it is not possible to create duplicates for the same document. However, using the document content indexing options above, it is possible to set up each Workgroup Server to process a unique set of records by either using the date fields to specify a range or a URI range.
   Below is an example Oracle script that could be used to divide the data up into three groups of URIs based on the size of the electronic document stores. This would ensure that they share the load evenly. This script could be modified for the other databases or to break up the data into the number of groups required.

       Set serveroutput on;

       DECLARE

       CurrentValue NUMBER := 0;

       RunningSum NUMBER := 0;

       MaxLimit NUMBER := 0;

       MaxUri NUMBER := 0;

       LowerUri NUMBER := 0;

       UpperUri NUMBER := 0;

       CurrentUri NUMBER := 0;

       bSet BOOLEAN;

       BEGIN

       SELECT Round(Sum(reBytes)/3)

       INTO MaxLimit

       FROM TSRECELEC;

       SELECT Max(uri)

       INTO MaxUri

       FROM TSRECELEC;

       FOR i IN 1..3

       LOOP

       RunningSum := 0;

       bSet := TRUE;

       LowerUri := 0;

       UpperUri := 0;

       LOOP

       SELECT uri, reBytes

       INTO UpperUri, CurrentValue

       FROM ( SELECT * FROM TSRECELEC

       WHERE uri > CurrentUri

       ORDER BY uri)

       WHERE ROWNUM <= 1;

       IF bSet THEN

       LowerUri := UpperUri;

       bSet := FALSE;

       END IF;

       RunningSum := RunningSum + CurrentValue;

       CurrentUri := UpperUri;

       IF CurrentUri = MaxUri THEN

       EXIT;

       ELSIF RunningSum < MaxLimit THEN

       CONTINUE;

       ELSE

       EXIT;

       END IF;

       END LOOP;

       DBMS_OUTPUT.PUT_LINE ('WGS ' || TO_CHAR(i) ||' Min Uri = ' || TO_CHAR(LowerUri));

       DBMS_OUTPUT.PUT_LINE ('WGS ' || TO_CHAR(i) ||' Max Uri = ' || TO_CHAR(UpperUri));

       END LOOP;

       END;

   When employing this approach, it is important to ensure that the IDOL Server has been correctly sized to handle the store content.

Related information

Copyright 2008-2024 Open Text
Version 24.4 | Last updated October 2024
Contact

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to hprmdocumentation@microfocus.com.

We appreciate your feedback!

Product: Content Manager Enterprise Studio 24.4

Topic Title: Re-indexing the dataset using IDOL