Knowledge Discovery 25.1
Getting Started Guide

# Configure Languages

The Content component can process documents in multiple languages and encodings. For each language that you want to use, you must define the language types in the Content component configuration file. You must also configure the Content component to classify documents, either by automatically detecting the language and encoding, or by reading the language type from a field.

## Define Language Types

To run the Content component in multiple languages, specify the language types you want the Content component to process. A language type is a combination of the language and encoding.

NOTE: You must specify languages and language types before you index data into the Content component.

To define language types

1. Open the Content component configuration file in a text editor.

2. Find the `[LanguageTypes]` section. List the languages that you want the Content component to process. You must use ASCII characters to specify the language names.

   For example:

       [LanguageTypes]
       0=English
       1=Afrikaans
       2=General

3. For each language, create a configuration section that matches the name you defined in the `[LanguageTypes]` section.

4. In this section, specify appropriate settings that determine how the Content component handles this language. For details on the configuration parameters you can use, refer to the Content component Reference.

5. For each section, set the `Encodings` parameter to a list of the encodings and corresponding language types used by the language. List each encoding and language in the format `encoding:languagetype`. Separate multiple language types with commas.

   For example:

       [english]
       Encodings=ASCII:englishASCII,UTF8:englishUTF8
       Stoplist=english.dat
       IndexNumbers=1

       [afrikaans]
       Encodings=ASCII:afrikaansASCII,UTF8:afrikaansUTF8
       IndexNumbers=1

       [general]
       Encodings=UTF8:generalUTF8,ASCII:generalASCII,CYRILLIC:generalC
       YRILLIC
       IndexNumbers=1

6. Save and close the configuration file.

7. Restart Content for your changes to take effect.

You can now configure the Content component to associate the language types you defined with documents.

## Associate Language Types with Documents

After you define all the language types you want the Content component to process, set up a field process that allows the Content component to associate these language types with documents.

The way you configure the field process depends on the documents that you want to index:

- If all the documents contains a field that exactly specifies the language type, configure a field process to define this field as a `LanguageType` field. The language types that appear in this field must exactly match the language types that you define in the `[LanguageTypes]` configuration section. See Configure Fields.

- If the documents contain a field that specifies the language, but does not exactly specify the language type, you can configure field processes to detect the language from this field data.

To configure the Content component to detect the language type from field data

1. Open the Content component configuration file in a text editor.

2. In the `[FieldProcessing]` section, define a field process for each language that you want to detect.

   For example:

       [FieldProcessing]
       0=DetectArabic
       1=DetectEnglish
       2=DetectFrench

3. Create a configuration section with the same name as each of the field processes you defined in the `[FieldProcessing]` section.

4. In this section:

   - Set `Property` to the name of the property for the specified language type.

   - Set `PropertyFieldCSVs` to a comma-separated list of fields that can contain the language data.

   - Set `PropertyMatch` to a comma-separated list of values that this field might contain to identify the specified language type.

   For example:

       [DetectArabic]
       Property=SetArabicProperty
       PropertyFieldCSVs=*/DRELANGUAGETYPE,*/LANG
       PropertyMatch=arabic

       [DetectEnglish]
       Property=SetEnglishProperty
       PropertyFieldCSVs=*/DRELANGUAGETYPE,*/LANG
       PropertyMatch=*eng*,uk,*british

       [DetectFrench]
       Property=SetFrenchProperty
       PropertyFieldCSVs=*/DRELANGUAGETYPE,*/LANG
       PropertyMatch=*fre*,fran*

5. Create a configuration section for each property that you define in the field processing sections.

6. In the property configuration section, set the `LanguageType` parameter to the language type to use to define documents that match this property (that is, that contain a field with a matching value for the field process). This language type must match one of the language types you configure in the `[LanguageTypes]` configuration section.

   For example:

       [SetArabicProperty]
       LanguageType=Arabic

       [SetEnglishProperty]
       LanguageType=English

       [SetFrenchProperty]
       LanguageType=French

7. Save and close the configuration file.

8. Restart the Content component for your changes to take effect.

Copyright 2024 Open Text
Version 25.1 | Last updated January 2025

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: Knowledge Discovery 25.1

Topic Title: Configure Languages