NiFi Ingest 12.13

# QueryIDOL

A processor that sends a query to an IDOL Content component and outputs the resulting documents as FlowFiles.

You can specify a query in one of the following ways:

- Define a query using the Text, Field Text, Database Match, and other relevant properties in the processor configuration. With this approach you can define a single, pre-configured query and there is no incoming connection. The processor sends a `Query` action to IDOL Content each time it is scheduled to run, so Micro Focus recommends that you check the scheduling settings and configure a reasonable interval between each cycle. You can choose whether the second and subsequent queries return all matching documents, or only new documents, by setting the property New documents only.
- Initiate a query by routing a FlowFile to the processor. The query parameters (including text, field text, and database name) can be set as FlowFile attributes. In this case, configure the processor properties using NiFi expression language that references the FlowFile attributes. The incoming FlowFile is routed to the "original" relationship, with additional attributes that you can use to construct a new query to retrieve new documents (the same query performed later could return additional results, if new documents are indexed). In this case the scheduling options define how often the processor checks its incoming connection for queued FlowFiles.

This processor can be useful when you want to re-ingest some of the documents in your IDOL index. You can send a query to IDOL Content and then route the result documents back to an IDOL Connector so that the connector retrieves those documents again.

NOTE: The connector must support the synchronize from identifiers feature.

NOTE: Your query results might contain documents retrieved by many different connectors. Each document must be routed back to the correct connector. You can route the resulting FlowFiles by the value of the `idol.doc.source` attribute (for a description of this attribute, see Introduction to FlowFiles and Documents). Apache NiFi includes a built-in processor named `RouteOnAttribute` that can route FlowFiles based on attribute values.

## Properties

Name | Default Value | Description
--- | --- | ---
IDOL License Service |  | An IdolLicenseServiceImpl that provides a way to communicate with an IDOL License Server.
IDOL Host |  | The host name or IP address of the IDOL Content component.
IDOL ACI Port |  | The ACI port of the IDOL Content component.
SSL Config Service |  | An optional IdolSSLConfigServiceImpl that specifies the settings to use to communicate over SSL/TLS. Set this property only if your IDOL Content component has been configured to accept connections over SSL.
Batch Size | 100 | The number of documents to release to the output relationship at one time.
Text |  | Sets the value of the `Text` parameter in the `Query` action.
Field Text |  | Sets the value of the `FieldText` parameter in the `Query` action.
Database Match |  | Sets the value of the `DatabaseMatch` parameter in the `Query` action.
Print | AllSections | Sets the value of the `Print` parameter in the `Query` action.
Print Fields | AUTN_SOURCE, AUTN_IDENTIFIER | Sets the value of the `PrintFields` parameter in the `Query` action.
New documents only | True | Specifies whether to return only new documents. When you set this property to FALSE, the processor outputs a FlowFile for every document in the IDOL index that matches the query. (With an incoming connection, this instructs the processor to ignore the `idol.query.result.lastdocid` attribute in incoming FlowFiles - see the description of the original relationship, below). When you set this property to TRUE, the processor outputs a FlowFile for every document in the IDOL index that matches the query, but ignores documents that were returned on previous cycles. (With an incoming connection, requires you to set the attribute `idol.query.result.lastdocid` on incoming FlowFiles).
XML reference path | DREREFERENCE | The field, in documents in the index, that contains the document reference.
XML source path | AUTN_SOURCE | The field, in documents in the index, that contains the document source.
XML identifier path | AUTN_IDENTIFIER | The field, in documents in the index, that contains the document identifier.
XML content path | DRECONTENT | The field, in documents in the index, that contains the document content.
Dynamic properties |  | Any dynamic properties that you set are added to the `query` action as action parameters. For information about the action parameters that you can use, refer to the IDOL Content Component Reference. NOTE: You cannot set the action parameters `Combine`, `StoreState`, `StateMatchID`, `MinID`, or `MaxResults` because these are set automatically.
attr:name |  | When you configure the processor to run queries in response to incoming FlowFiles, you might want to add information about a query to all of the results. To do this, add dynamic properties named `attr:name`, where `name` is the name of a FlowFile attribute to add to each result FlowFile. The value of the dynamic property sets the value, and can be a constant or can use NiFi expression language to copy a FlowFile attribute from the incoming FlowFile.

## Relationships

Name | Description
--- | ---
success | FlowFiles that represent documents returned from the query action that was sent to the IDOL Content component.
original | This relationship is used only when the processor has an incoming connection. Incoming FlowFiles that initiate a query are routed to this relationship. The processor adds the following attributes to each processed FlowFile: `idol.query.result.lastdocid` - the largest document ID returned by the query. You can set this attribute in FlowFiles that you route to the QueryIDOL processor, to retrieve only new documents (the same query performed later could return additional results, if new documents are indexed). `idol.query.result.hitcount` - the total number of documents routed to the success relationship.

## View and Clear State Information

When there is no incoming connection, the processor stores state information so that it can obtain only new documents. If you change the query that you have configured, Micro Focus recommends clearing the state information.

To clear the state information

1. Stop the processor.

2. Right-click the processor and click View State.

   A dialog box opens.

3. Click Clear state and then click CLOSE.

## Stored State Tokens

When a query is complete the processor uses the `TokenManagement` action to delete the state token in IDOL Content. This is an administrative action and fails unless NiFi is authorized to use it. There is no effect on NiFi Ingest if the `TokenManagement` action fails, but the state token is not deleted.

© Copyright 2018-2022 Micro Focus or one of its affiliates
Version 12.13 | Last updated October 2022

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to swpdl.idoldocsfeedback@microfocus.com.

We appreciate your feedback!

Product: IDOL NiFi Ingest 12.13

Topic Title: QueryIDOL