NiFi Ingest 25.1
NiFi Ingest

# QueryIDOL

A processor that sends a query to a Content componentand outputs the resulting documents as FlowFiles.

You can specify a query in one of the following ways:

- Define a query using the Text, Field Text, Database Match, and other relevant properties in the processor configuration. With this approach you can define a single, pre-configured query and there is no incoming connection. The processor sends a `Query` action to Content each time it is scheduled to run, so OpenText recommends that you check the scheduling settings and configure a reasonable interval between each cycle. You can choose whether the second and subsequent queries return all matching documents, or only new documents, by setting the property New documents only.
- Initiate a query by routing a FlowFile to the processor. The query parameters (including text, field text, and database name) can be set as FlowFile attributes. In this case, configure the processor properties using NiFi expression language that references the FlowFile attributes. The incoming FlowFile is routed to the "original" relationship, with additional attributes that you can use to construct a new query to retrieve new documents (the same query performed later could return additional results, if new documents are indexed). In this case the scheduling options define how often the processor checks its incoming connection for queued FlowFiles.

This processor can be useful when you want to re-ingest some of the documents in your index. You can send a query to Content and then route the result documents back to a connector so that the connector retrieves those documents again.

NOTE: The connector must support the synchronize from identifiers feature.

NOTE: Your query results might contain documents retrieved by many different connectors. Each document must be routed back to the correct connector. You can route the resulting FlowFiles by the value of the `idol.doc.source` attribute (for a description of this attribute, see Introduction to FlowFiles and Documents). Apache NiFi includes a built-in processor named `RouteOnAttribute` that can route FlowFiles based on attribute values.

## Properties

Name | Default Value | Description
--- | --- | ---
IDOL License Service |  | An IdolLicenseServiceImpl that provides a way to communicate with a Knowledge Discovery License Server.
IDOL Host |  | The host name or IP address of the Content component.
IDOL ACI Port |  | The ACI port of the Content component.
SSL Config Service |  | An optional IdolSSLConfigServiceImpl that specifies the settings to use to communicate over SSL/TLS. Set this property only if your Content component has been configured to accept connections over SSL.
Batch Size | 100 | The number of documents to release to the output relationship at one time.
Text |  | Sets the value of the `Text` parameter in the `Query` action.
Field Text |  | Sets the value of the `FieldText` parameter in the `Query` action.
Database Match |  | Sets the value of the `DatabaseMatch` parameter in the `Query` action.
Print | AllSections | Sets the value of the `Print` parameter in the `Query` action.
Print Fields | AUTN_SOURCE, AUTN_IDENTIFIER | Sets the value of the `PrintFields` parameter in the `Query` action.
New documents only | True | Specifies whether to return only new documents. When you set this property to FALSE, the processor outputs a FlowFile for every document in the index that matches the query. (With an incoming connection, this instructs the processor to ignore the `idol.query.result.lastdocid` attribute in incoming FlowFiles - see the description of the original relationship, below). When you set this property to TRUE, the processor outputs a FlowFile for every document in the index that matches the query, but ignores documents that were returned on previous cycles. (With an incoming connection, requires you to set the attribute `idol.query.result.lastdocid` on incoming FlowFiles).
XML reference path | DREREFERENCE | The field, in documents in the index, that contains the document reference.
XML source path | AUTN_SOURCE | The field, in documents in the index, that contains the document source.
XML identifier path | AUTN_IDENTIFIER | The field, in documents in the index, that contains the document identifier.
XML content path | DRECONTENT | The field, in documents in the index, that contains the document content.
Dynamic properties |  | Any dynamic properties that you set are added to the `query` action as action parameters. For information about the action parameters that you can use, refer to the Content Component Help. NOTE: You cannot set the action parameters `StoreState`, `StateMatchID`, or `MinID` because these are set automatically. The `Combine` parameter has a default value of "simple" but can be overridden. The `MaxResults` parameter has a default value of 10000000 but can be overridden.
attr:name |  | When you configure the processor to run queries in response to incoming FlowFiles, you might want to add information about a query to all of the results. To do this, add dynamic properties named `attr:name`, where `name` is the name of a FlowFile attribute to add to each result FlowFile. The value of the dynamic property sets the value, and can be a constant or can use NiFi expression language to copy a FlowFile attribute from the incoming FlowFile.

## Relationships

Name | Description
--- | ---
success | FlowFiles that represent documents returned from the query action that was sent to the Content component.
original | This relationship is used only when the processor has an incoming connection. Incoming FlowFiles that initiate a query are routed to this relationship. The processor adds the following attributes to each processed FlowFile: `idol.query.result.lastdocid` - the largest document ID returned by the query. You can set this attribute in FlowFiles that you route to the QueryIDOL processor, to retrieve only new documents (the same query performed later could return additional results, if new documents are indexed). `idol.query.result.hitcount` - the total number of documents routed to the success relationship.

## View and Clear State Information

When there is no incoming connection, the processor stores state information so that it can obtain only new documents. If you change the query that you have configured, OpenText recommends clearing the state information.

To clear the state information

1. Stop the processor.

2. Right-click the processor and click View State.

   A dialog box opens.

3. Click Clear state and then click CLOSE.

## Stored State Tokens

When a query is complete the processor uses the `TokenManagement` action to delete the state token in Content. This is an administrative action and fails unless NiFi is authorized to use it. There is no effect on NiFi Ingest if the `TokenManagement` action fails, but the state token is not deleted.

Copyright 2018-2024 Open Text
Version 25.1 | Last updated January 2025

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: Knowledge Discovery NiFi Ingest 25.1

Topic Title: QueryIDOL