Content Manager Enterprise Studio 25.1 Enterprise Studio

# Advanced configuration of OEM IDOL

When installing Content Manager, you are also installing an OEM version of Autonomy IDOL. While the default settings work, it is highly recommended to secure and optimize your setup to suit your organization's needs.

If you are using your own implementation of Enterprise IDOL, you can disregard the content of this topic. To set up your working implementation of Enterprise IDOL to work with Content Manager, install and configure the Content Manager CFS connector as described in CM24.4_IDOL_DCI_Install_Config.pdf.

You can use the Content Manager IDOL Service configuration file, by default C:\Micro Focus Content Manager\IDOL\TRIM IDOL Service\TRIM IDOL Service.cfg, to set a large number of IDOL server configuration options. While these are not mandatory, you may find some useful. At a minimum, you should secure your server from unauthorized access.

Additionally, you can configure the IDOL content engine by using the configuration file, which by default are C:\Micro Focus Content Manager\IDOL\TRIM Content Service\TRIM Content Service.cfg

After making changes to the configuration files, make sure to save your changes, and then restart the respective service.

Securing the IDOL server

You can use an Internet browser to send index commands to the IDOL server, including delete, backup and initialization commands. While the OEM IDOL database itself is encrypted, by default, any client computer can be used to send index commands. Therefore, it is important you restrict access to the server to the administrator computers:

In the Content Manager IDOL Service configuration file, you will find the section:

    [Server]
    QueryClients=*.*.*.*
    AdminClients=*.*.*.*
    IndexClients=*.*.*.*

Instead of leaving the IP range for IndexClients open, restrict them to the IP addresses of the computers who need to be able to send index commands. You can specify a range or specific computers:

Range:

    IndexClients=15.146.146.*

Specific computers, separated by commas with no spaces:

    IndexClients=localhost,myIDOLserver,127.0.0.1

Creating additional content engines

By default, the Content Manager IDOL installation creates a content engine. For medium to large installations, it may be necessary to add additional content engines.

These are the manual instructions to do so, which are not recommended. Instead, you should use the instructions for creating additional content services by using the IDOL components .msi installation file, starting in the topic Content Index Properties dialog box.

This example creates a third Content Manager IDOL content service. For each additional content service, replace the number 3 with the appropriate number and pick suitable unused port numbers.

1. Before creating additional content engine services, stop all Content Manager IDOL services
2. Navigate to C:\Program Files\Micro Focus\Content Manager\IDOL and copy the folder TRIM Content Service 1 to a temporary folder
3. Rename the new folder to TRIM Content Service 3
4. In the folder TRIM Content Service 3, rename the files TRIM Content Service 1.cfg and TRIM Content Service 1.exe to TRIM Content Service 3.cfg and TRIM Content Service 3.exe respectively
5. Edit the file TRIM Content Service 3.cfg and change the sections Service Settings and Server Settings to the following:

       //--------------------------- Service Settings -----------------//

       [Service]

       ServicePort=9302

       ServiceStatusClients=*.*.*.*

       ServiceControlClients=*.*.*.*

       //--------------------------- Server Settings ------------------//

       [Server]

       Port=9300

       IndexPort=9301

   You can use any value for Port, IndexPort and ServicePort number that is available in your environment. However, what you use must be reflected correctly in the Content Manager IDOL configuration file TRIM IDOL Service.cfg.

6. Change the sections Path Settings and Logging Settings to the following:

       //--------------------------- Path Settings --------------------//

       [Paths]

       BitFieldPath=D:\Content Manager\IDOL\TRIM Content Service 3\bitfield

       DyntermPath=D:\Content Manager\IDOL\TRIM Content Service 3\dynterm

       IndexQueuePath=D:\Content Manager\IDOL\TRIM Content Service 3\indexqueue

       IndexTempPath=D:\Content Manager\IDOL\TRIM Content Service 3\indextmp

       MainPath=D:\Content Manager\IDOL\TRIM Content Service 3\main

       NodetablePath=D:\Content Manager\IDOL\TRIM Content Service 3\nodetable

       NumericPath=D:\Content Manager\IDOL\TRIM Content Service 3\numeric

       RefIndexPath=D:\Content Manager\IDOL\TRIM Content Service 3\refindex

       SecIndexPath=D:\Content Manager\IDOL\TRIM Content Service 3\secindex

       SortfieldPath=D:\Content Manager\IDOL\TRIM Content Service 3\sortfield

       StatusPath=D:\Content Manager\IDOL\TRIM Content Service 3\status

       StatePath=D:\Content Manager\IDOL\TRIM Content Service 3\storedstate

       TagPath=D:\Content Manager\IDOL\TRIM Content Service 3\tagindex

       LanguageDirectory=..\shared\langfiles

       //--------------------------- Logging Settings -----------------//

       [Logging]

       LogDirectory=D:\Content Manager\IDOL\TRIM Content Service 3\logs

   In this example, the path for the IDOL indexes is D:\Content Manager\IDOL. You can change it to any location on any server by changing it in the configuration file TRIM IDOL Service.cfg. You can also use UNC paths.

7. Save the configuration file
8. Copy the entire directory TRIM Content Service 3 to C:\Program Files\Micro Focus\Content Manager\IDOL
9. Navigate to the directory C:\Program Files\Micro Focus\Content Manager\IDOL\TRIM IDOL Service and as administrator, edit the file TRIM IDOL Service.cfg:
   - Set:

         [DAHEngines]

         Number=3

   - Add:

         [DAHEngine2]

         Host=servername

         Port=9300

10. Repeat for the [DIHEngines]:
    - Set:

          [DIHEngines]

          Number=3

    - Add:

          [DIHEngine2]

          Host=servername

          Port=9300

      Ensure the port number matches the one that you used in the file TRIM Content Service 3.cfg.

11. Save TRIM IDOL Service.cfg
12. Open a command prompt as administrator and navigate to the directory C:\Program Files\Micro Focus\Content Manager\IDOL\TRIM Content Service 3.
13. Run the command Trim Content Service 3.exe –install.

    This will install Content Manager Content Service 3 as a service.

14. Open Windows Services and restart the Content Manager IDOL Service, Content Manager IDOL Content Service 1, Content Manager IDOL Content Service 2 and Content Manager IDOL Content Service 3.

    For the new service, you may choose to change the value for Log on as.
    Please note the name of the new service does not start with HP.

15. Navigate to the directory set as the path for Content Manager IDOL Content Service 3 and confirm that the IDOL database structure has been created.
    If it has, you have successfully installed the additional content engine.
    If it has not, verify that you followed the steps above to the letter and correct them where needed.

IDOL options

There are many options that can be applied to a Content Manager IDOL installation to optimize the IDOL implementation and to tailor the solution to meet your requirements. Here are some of the options to consider when building your IDOL environment.

Threads

Threads applies to the Content Manager IDOL service and sets the maximum number of parallel actions that can be handled by an IDOL Server and the number of action commands that the server can process in parallel at any one time. You can calculate the optimum number of threads as follows:

(1 x number of cores ) + 1 spare thread.

You can set the option in the configuration file TRIM IDOL Service.cfg, under the [server] section. The default is 4.

Example:

    Threads=9

This option is not identical with the Content Manager property Number of processing threads that you can set when reindexing in the IDOL Record Reindex dialog box, Options tab.

Maximum queued transactions

Content Manager sends the transaction and command files to all content engines when it has processed a certain number of documents. That number of documents is determined by the value of Maximum queued transactions in the Content Index Properties dialog box. See Content Index Properties dialog box.

DistributeByReference

Plain non-mirror mode sends all commands to all engines and lets the child engines determine which documents to index. With reference-based indexing, DIH performs these calculations, reducing network traffic and load on the child servers.

In conjunction with the previous option, you can configure IDOL to send each content service only part of the transaction file. To do so, in the IDOL service configuration file TRIM IDOL Service.cfg, [server] section, add:

    DistributeByReference=true

DCI round-robin

An alternative to DistributeByReference is to use a round-robin approach to the processing of transaction files in IDOL. Rather than breaking up the transaction file and distributing it to the content engines, round-robin sends the transaction file to one engine at a time and cycles through the available content engines.

The advantage here is that the Distributed Index Handler (DIH) does not need to break up the transaction file or send it to all content engines.

See the IDOL DIH 7.3 Administration Guide at http://support.openview.hp.com/selfsolve/document/KM00204534 for more information.

MaxResults

The MaxResults setting in the Content Manager IDOL configuration files limits how many results IDOL will return to Content Manager in a single search. This is included as a performance consideration.

Change the value, for example to:

    MaxResults=100000

By default, the value is 10000.

To change the figure, edit the TRIM Content Service [number].cfg configuration files, by default in C:\Program Files\Micro Focus\Content Manager\IDOL\TRIM Content Service [number] and the TRIM IDOL Service.cfg configuration file, by default in C:\Program Files\Micro Focus\Content Manager\IDOL\TRIM IDOL Service.

In the TRIM Content Service [number].cfg files, you will find MaxResults under [Server].

In the file TRIM IDOL Service.cfg, you will find it under [DistributionSettings].

The value to choose for MaxResults depends on your environment and performance expectations:
A lower value means that IDOL will return fewer results, but faster.
A higher value means that IDOL will return more results, but it will take longer, which in turn may cause the Content Manager Workgroup Server to time out before IDOL has responded.

Therefore, if you are using a higher value for MaxResults, you should probably also increase the time before a query times out by using the Content Manager Enterprise Studio option IDOL Connection Timeout. See Content Index Properties dialog box.

MaxSyncDelay

MaxSyncDelay in the TRIM IDOL Service.cfg configuration file, [Server] section, and TRIM Content Service [number].cfg, enables you to specify the maximum time in seconds that can elapse before the IDOL Server is forced to store index data on disk.

This option is particularly useful when running a reindex, or for the initial indexing of the document store. As the content services databases grow, index data processing to disk takes more time.

This setting increases the time it takes for the information to be available for searching. However, during reindexing, write performance is more important than read performance.

By default, the value is 120.

For reindexing, it is recommended to set it to 360 to 720 seconds:

    MaxSyncDelay=360

Set the value in all relevant configuration files.

MaxSyncDelay requires the DelayedSync option in the same configuration file to be set to true. For Content Manager OEM IDOL, the default is true.

When set to false, the IDOL server stores index data every time it finishes indexing an IDX or XML file, or when the specified value has been reached.

Enter true for the IDOL Server to only store index data on disk when the index cache is full, or when the specified value has been reached:

    DelayedSync=true

Timeout

The value for Timeout controls how long the main IDOL Server will wait for an action to be completed.

    Timeout=60

EngineTimeout

The value for EngineTimeout controls how long the main service will wait for a content engine to return its results.

    EngineTimeout=60

StateTokens

There are two settings in the TRIM IDOL Service.cfg file, section Distributed Settings, that manage StateTokens in IDOL:

- StoredStateTokenExpiryPeriod
- StoredStateTokenLifetime

The setting for StoredStateTokenLifetime is 0 by default and you should not change this setting.

    StoredStateTokenExpiryPeriod=300

IndexCacheMaxSize

This setting in the TRIM Content Service [number].cfg configuration files determines the amount of index data to store in the cache before writing it to disk. Increasing the size of the cache can improve indexing performance. However, you should ensure that the host computer has sufficient memory available. The default value is 102400, which refers to kbytes.

To improve performance during reindexing, increase this value to 612000 to 1024000, which is 512MB to 1GB. This setting works with the option MaxSyncDelay, and the content service will write index data to disk only when either the delay timer is reached or the cache is full. This setting is in the [IndexCache] section:

    IndexCacheMaxSize=102400

NOTE: You can dynamically grow or shrink the indexing cache after IDOL Server startup by using the DRERESIZEINDEXCACHE index command.

For example, to dynamically resize the index cache and increase the sync delay, use the following command in a browser on the IDOL Server:

http://localhost:9001/DRERESIZEINDEXCACHE?IndexCacheMaxSize=1024000&maxSyncDelay=360

StopwordIndex

By default, IDOL does not index or search on a number of operator words, such as AND, NOT, TO, DO, NEAR, MORE, etc. You can enable indexing of these operator words by editing the IDOL configuration files. If you are changing this option after the index has already been created, you should perform a full Content Managerdocument content reindex.

In the Content Manager IDOL Service configuration file and each of the Content Manager Content Service configuration files, in the section [LanguageTypes], add:

    StopwordIndex=1

Paths

The default location for the content engines and log files is determined by the settings in the TRIM IDOL Service.cfg file and the respective Content Manager Content Service configuration file. It is C:\Micro Focus Content Manager by default.

To specify a different location for the Content Manager IDOL log files, insert the following [paths] statement into the file TRIM IDOL Service.cfg:

    [Paths]

    Incoming=D:\Content Manager\IDOL\TRIM IDOL Service\incoming

    Archive=D:\Content Manager\IDOL\TRIM IDOL Service\archive

    Failed=D:\Content Manager\IDOL\TRIM IDOL Service\failedTasks

This is particularly relevant for systems that have limited space on the C: drive. In the event of a problem, the failed directory receives all unprocessed transaction files and this may fill the drive.

Database Read-only

You can set a Content Manager content engine to read-only by adding the following to the Content Manager Content Service configuration file in the respective content service in C:\Program Files\Micro Focus\Content Manager\IDOL:

    [CM_PM]

    DatabaseReadOnly=TRUE

PM represents the database ID of the Content Manager dataset.

Default Operator

The default IDOL operator is not strictly an OR, but similar to an OR with calculations that affect relevancy sorting. To change this, in the file TRIM Content Service [number].cfg, [Server] section, add DefaultQueryOperator=<Operator>.

Example:

    DefaultQueryOperator=AND

Logs

Logs relevant to the content engines are by default in C:\Micro Focus Content Manager\IDOL\TRIM Content Service [number]\logs. There are application.log, index.log and query.log.

Logs relevant to the Content Manager IDOL services including DIH and DAH logs are by default in C:\Micro Focus Content Manager\IDOL\TRIM IDOL Service\logs.

Location of the transaction files and working directory for opening compressed files during text extraction:

C:\Users\<services_user>\AppData\Local\Micro Focus\Content Manager

C:\Micro Focus Content Manager\ServerLocalData\TRIM\Log on the Workgroup Servers holds all relevant Workgroup Server, Event Server and document content indexing logs.

Changing the log file locations

In the default configuration, the log files are located in C:\Micro Focus Content Manager. You can change this by changing the TRIM Content Service.cfg configuration files in C:\Program Files\Micro Focus\Content Manager\IDOL\TRIM Content Service 1 and 2. The path to the log files is in the section [Logging]:

    //--------------------------- Logging Settings --------------------//

    [Logging]

    LogDirectory=C:\Micro Focus Content Manager\IDOL\TRIM Content Service 1\logs

    LogTime=TRUE

    LogOutputLogLevel=TRUE

    LogLevel=Normal

Change the directory to your desired location.

In the Content Manager IDOL Service configuration file at C:\Program Files\Micro Focus\Content Manager\IDOL\TRIM IDOL Service.cfg, there is a similar path:

    //---------------------------Logging-------------------------------//

    [Logging]

    LogDirectory=C:\Micro Focus Content Manager\IDOL\TRIM IDOL Service\logs

    LogTime=TRUE

Change this path, as well.

Searching options
AugmentSeparators

By default, IDOL uses certain special characters when users run a search for document content, which effect the search result for terms that contain these characters. IDOL therefore does not index these characters when they appear in the document content. Therefore, it is good to be aware of them or, if required, modify the list that IDOL uses. The list is in the Content Manager IDOL Service Configuration file under Languages types [English]:

    AugmentSeparators=-–—@“”‘’_.…

The most significant here are the at symbol (@), comma (,), dash (-) and full stop (.). A search for a word or string that contains these characters, for example a search for TPS-1234 is treated as a search for TPS or 1234. Therefore, although Content Manager will return the documents with TPS-1234, it will also return other records containing only TPS or 1234. To search for TPS-1234 exactly, you would put the text within quotation marks, like “TPS-1234”.

The same applies to an email address: Searching for John.Smith@hpe.com would become a search for John or Smith or hpe or com. To find the email address exactly, you would search for “John.Smith@hpe.com”.

The same applies to the use of wildcard characters. A search for *ps-1234 would return all records with a word ending in ps, for example soups, troops and groups. If you would rather find the records with ps-1234 exactly, you can put the text within quotation marks as in “*ps-1234”.

To have IDOL index these special characters, remove them from the list of AugmentSeparators. If you are changing the setup of an existing index, this will require a full reindex.

IDOL stop words

Content Manager IDOL uses a default set of stop or noise words when building the document content index, which IDOL does not index when they appear in document content. These noise words do not relate to the set of noise words in Content Manager that are applied to metadata fields such as record titles and notes.

The IDOL noise words are stored in .dat files in the directory C:\Program Files\Micro Focus\Content Manager\IDOL\shared\langfiles. By default, they contain common English words.

After modifying the list of noise words, you should reindex the dataset for the new settings to take effect.

Setting up IDOL to index Additional Fields

Content Manager IDOL can index the content of Additional Fields as metadata. To configure it to do so, you need to insert the Additional Field in the configuration files as a two-letter prefix that indicates the field type, followed by an underscore and the Additional Field name without any spaces.
For example, for an Additional Field of type integer called Batch number, it would be TN_BATCHNUMBER.
The prefixes:

- TN_ - represents the types integer, currency, decimal and large number
- TD_ - represents the types date and date and time
- TS_ - represents the types string and Location

Insert your TN_ and TS_ Additional Field code in the TRIM Content Service.cfg configuration file in the [SetIndexFields] block, parameter PropertyFieldCSVs, with */ in front of it.

Insert your TD_ Additional Field code in the block [SetDateFields] instead.

After modifying the list of indexed Additional Fields here, you should reindex the dataset metadata for the fields to be added to the index.

Related information

Copyright 2008-2025 Open Text
Version 25.1 | Last updated October 2024
Contact

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to hprmdocumentation@microfocus.com.

We appreciate your feedback!

Product: Content Manager Enterprise Studio 25.1

Topic Title: Advanced configuration of OEM IDOL