Knowledge Discovery 25.1
Knowledge Discovery Expert

# Languages

When the Content component processes a document, it treats the text as a series of tokens (words), each of which is a unit of meaning. At a low level, this method is language independent. However, you can improve your query results by applying some language dependent processing.

Language dependent configuration allows you to:

- make sure that all your content is treated consistently, allowing cross-lingual search.

- filter your searches to content in a specific language.

This section describes the most important language concepts, and explains why you might use them.

- Language Types. The language and encoding of a document.

- Tokenization. The methods Content uses to split text into searchable tokens.

- Stemming. Processing that reduces groups of related words to a common stem.

- Stop Lists. Lists of words that do not convey meaning in documents.

- Cross-Lingual Search. Search across documents in multiple languages.

- Order of Language Processes. The order in which the language processing steps occur during indexing and querying.

Copyright 2024 Open Text
Version 25.1 | Last updated January 2025

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: Knowledge Discovery 25.1

Topic Title: Languages