IDOL 24.4
IDOL Expert

# Eduction

Eduction is the process of finding entities in documents, and typically extracting them to form additional fields in the document. Entities are words, phrases, or blocks of information. Eduction finds entities by using predefined or custom grammars, which use linguistic, regular expression, or term-based rules to define the pattern to match. Common entities include names, addresses, phone numbers, and credit card numbers.

Eduction allows you to automatically extract entities from your documents and add them as metadata to your documents before indexing into IDOL Server. You can extract common search phrases from documents before indexing, and tag the documents with this data. You can then use the tags to make it very quick and easy to search for these common values. It is therefore a useful tool for preprocessing data.

Eduction is most commonly used in IDOL Server as a plug-in module for pre-index processing. It is also available in a stand-alone command-line tool, an API (for the C++ and Java languages), or an ACI server. The command-line tool (edktool) is used for compiling grammars and testing, in addition to extraction.

## Grammar Files

Eduction has a large and growing collection of standard grammar files, which allow you to quickly extract common entities. These are available for many major languages and geographical locations. You can set up Eduction to extract the information specified by the standard grammars very quickly.

In some cases you must write your own custom grammars, for example, if the standard grammar does not return the expected number of matches. See When to Extend a Grammar.

You can build custom grammars from the ground up. However, the standard grammars provide many basic entities that you can reference in your grammars, which allows you to create new custom grammars quickly.

For general information about grammar syntax and extensions, refer to the Eduction User and Programming Guide.

For a detailed tutorial that describes how to extend a grammar, see Eduction Grammar Tutorial.

## Configure Eduction

Eduction has many configuration options that allow you to change its matching behavior or choose which matches to return. When you configure Eduction, you must carefully consider the kind of data you have, the grammar that you want to use, and the results that you want.

For details of the available Eduction configuration, refer to the Eduction User and Programming Guide.

## Compiled Grammars

The standard grammar files are provided in a proprietary compiled format, .ECR. The compiled format ensures that entity extraction is very efficient and consistent.

You can also use the edktool command-line tool to compile your custom grammars to .ECR format, which improves the performance of extraction for these grammars. If you do not want to compile grammars, Eduction can use grammars in the original XML format. This method saves you the compilation time, but generally results in slower entity extraction.

Related Topics

- Sentiment Analysis

Copyright 2024 Open Text
Version 24.4 | Last updated September 2024

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: IDOL 24.4

Topic Title: Eduction