Content Component 26.1
Content Component Help

# Use KillDuplicates and Combine on ReferenceType Fields

When you instruct the Content component to eliminate duplicate document copies at index time using a specific ReferenceType field (by setting the KillDuplicates parameter in the Content component configuration file), it automatically uses any field listed for PropertyFieldCSVs alongside this ReferenceType field in the Content component configuration to eliminate duplicate document copies as well.

However, Content cannot use the same field for deduplication as for the Combine action parameter, because the Combine operation clashes (carried out at query time) with Content eliminating duplicate fields. This clash means that, if you want to eliminate duplicate document copies and use the Combine action parameter, you must set up separate ReferenceType fields for these processes.

Related Topics

- Prevent Duplicate Documents

To use KillDuplicates and Combine on ReferenceType fields

1. Open the Content component configuration file in a text editor.

2. In the `[FieldProcessing]` section, add two processes that identify ReferenceType fields (note that you must set up a field process to identify ReferenceType fields before you start indexing documents into Content). One of them is used to eliminate duplicate copies of documents, and the other one is used for the Combine operation.

   For example:

       [FieldProcessing]
       0=MyFirstProcess
       1=MySecondProcess
       2=SetUpReferenceFields
       3=SetUpMoreReferenceFields

3. Create a section for the processes that you added, and in each section, create a property for the respective process (you define the property later by setting one or more applicable configuration parameters). Identify the fields that you want to associate with each process.

   NOTE: The properties that you create must not have the same name as the processes.

   For example:

       [MyFirstProcess]
       Property=MyFirstProperty
       PropertyFieldCSVs=*/MyField,*/MySecondField

       [MySecondProcess]
       Property=MySecondProperty
       PropertyFieldCSVs=*/MyThirdField

       [SetUpReferenceFields]
       Property=ReferenceFields
       PropertyFieldCSVs=*/DREREFERENCE,*/URL

       [SetUpMoreReferenceFields]
       Property=MoreReferenceFields
       PropertyFieldCSVs=*/DRETITLE

4. Create a section for each of the properties and specify appropriate configuration settings for each. These configuration parameters define the processes that are applied to all the fields (or all documents that contain the fields) that you previously associated with the processes. For example:

       [MyFirstProperty]
       HiddenType=True

       [MySecondProperty]
       Index=True

       [ReferenceFields]
       ReferenceType=True
       TrimSpaces=True

       [MoreReferenceFields]
       ReferenceType=True
       TrimSpaces=True

5. Save and close the configuration file.

6. Restart the Content component for your changes to take effect.

After you index documents into Content, you can use, for example, the `*/DREREFERENCE` field to eliminate duplicate copies of documents. (Content then automatically also uses the `*/URL` field for deduplication because it is listed alongside `*/DREREFERENCE` for `PropertyFieldCSVs`.) This leaves you free to use the `*/DRETITLE` field for the Combine operation.

Copyright 2025 Open Text
Version 26.1 | Last updated January 2026

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: Knowledge Discovery Content Component 26.1

Topic Title: Use KillDuplicates and Combine on ReferenceType Fields