Knowledge Discovery 25.1
Knowledge Discovery Expert

# Disk Usage in the Content component

See Also: Content Internal Storage and Indexes

You can see the amount of disk space that Content uses by using the `DiskReport` action. Alternatively, you can view information about the disk usage of directories and files in the working directory for each component on the Disk Report tab of the Performance page in the Monitor section of Knowledge Discovery Admin.

The following sections describe how to reduce disk usage for particular indexes.

## Nodetable

You can dramatically reduce the amount of disk space required by the nodetable if there are any fields in the document that you do not actually need to store for retrieval.

If you retrieve the documents from a different system by using the reference, you can configure Content to index the body of the document (usually extracted as the `DRECONTENT` field) but not store it. In this case, operations that require the original document body (for example, highlighting, and the `GetContent` action) require additional processing by the application that uses Knowledge Discovery. For example, rather than sending a `GetContent` action to Content, it must request the document from the original repository, or through a Connector.

The File Content Extraction extraction process might pull out fields or information that your particular system does not require. In this case you can also discard these fields and not store them.

There are two mechanisms for discarding fields:

- Configure `CantHaveFieldCSVs` and `MustHaveFieldCSVs`. This option strips the data out of the document before Content indexes the document.

- Set `NodetableStoreContent` to `False`, and then use the `StoredType` property to specify fields that you want to store. This option results in Content indexing the data, but then discarding those fields rather than writing them to disk.

Some functionality is unavailable for non-stored content, but in some cases there are alternative methods that you can use. For more information, see Storage of Document Content.

Storing fields allows Content to retrieve the data without needing to go back to the document source, and can improve overall query performance. You can also regenerate stored fields if index validation fails, or if you need to change the field configuration after indexing.

You can use the `NodeTableCompression` configuration parameter to compress the documents in the nodetable on disk. In this case, Content compresses data in the nodetable directory before storing it, reducing the Content component disk footprint.

## Index Cache, Diskindex and Unstemmed

The main methods for reducing disk usage of these indexes attempt to reduce the amount of indexed data. These methods include:

- stop lists

- `IndexNumbers`, `IndexNumbersTruncateLength`, and `IndexNumbersMaxValue` configuration parameters

- number of index fields

Reducing the `TermSize` also reduces the disk usage for the `diskindex` and index cache, but normally this gain is minimal.

Reducing `TermSize` by 1 for an index with 1 million unique terms only reduces disk space usage by 1 MB.

You can further reduce the unstemmed disk space usage by using the `UnstemmedIndexNumbers` and `UnstemmedMinDocOccs` configuration options.

The AdvancedPlus option can contribute to the disk usage of the `diskindex`. To minimize the `diskindex` size, OpenText recommends that you turn this option off unless you particularly need the `SENTENCE` and `PARAGRAPH` matching.

## Other Indexes

For the Match, Numeric, and Parametric indexes, you can reduce disk usage only by limiting the amount of data indexed for those types.

For the Sort index, you can reduce disk usage by limiting the `SortFieldStorageLength` parameter.

Related Topics

- Determine the Tokens to Index

- Numeric and Alphanumeric Terms

Copyright 2024 Open Text
Version 25.1 | Last updated January 2025

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: Knowledge Discovery 25.1

Topic Title: Disk Usage in the Content component