Knowledge Discovery 25.1
Knowledge Discovery Expert

# Check How Data is Extracted and Processed

When you deploy connectors and CFS, check that the data is extracted and processed as expected. If necessary, you can customize the documents produced by CFS.

## Test File Content Extraction Filtering

CFS includes an application (`filtertest.exe`) that filters a file using File Content Extraction, and writes the output to a text file. You can take a sample file from your repository and see how the file is processed. This test can provide useful information about how you might need to configure your CFS.

## Check Document Content

When you start extracting data, configure CFS to output documents as files (you can output documents as XML or IDX files). You can then see how the data from a repository is used to build documents, and verify that the data is processed as expected.

For example:

- Check that the data you want to extract has been extracted successfully.

- Check that the data has been written to the correct fields. The field names that you use to store extracted data are particularly important for some Knowledge Discovery operations, for example Parametric Search .

- Check that the content is valid. If you notice that documents contain hexadecimal codes (for example `0x456`), it is likely that the original characters have been escaped.

  Text returned by File Content Extraction is encoded as UTF-8. File Content Extraction attempts to determine the encoding of files that are processed, but if it cannot determine the correct encoding, CFS escapes any characters that are not valid UTF-8.

## Reset a Connector

When testing connectors and CFS, you might want to index the same files many times and observe how the data is processed. However, connectors keep a record of the files that have been retrieved. A connector creates a datastore (`.db`) file for each fetch task that you run. To retrieve the same files again you must delete this record.

You can delete the datastore (`.db`) files in the connector installation folder, or set configuration parameters in the connector’s configuration file so that the connector does this automatically (see Configuration Parameters for Testing Connectors).

Related Topics

- File System Connector

Copyright 2024 Open Text
Version 25.1 | Last updated January 2025

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: Knowledge Discovery 25.1

Topic Title: Check How Data is Extracted and Processed