NiFi Ingest 25.2
NiFi Ingest

# Changes to your DataFlow

All of the nodes in a NiFi cluster run the same data flow. Any changes that you make to your data flow on one node are replicated on the other nodes.

TIP: Although a Connector appears in the data flow on all nodes, it only runs on a single node. The node is chosen automatically by NiFi. If that node becomes unavailable, another node takes over. Subsequent processors, such as KeyView Extraction and Filtering, run simultaneously on all of the nodes in your cluster.

The FlowFiles that a connector produces need to be distributed across all of the nodes in the cluster. This does not happen automatically. OpenText recommends that you configure the output connection from the connector and set the Load Balance Strategy to Round Robin.

To configure round-robin load balancing

1. Right-click the output queue (success relationship) for the connector and click Configure.

   The Edit Connection dialog box opens.

2. Click the Settings tab.
3. In the Load Balance Strategy list, click Round Robin.

To run a NiFi cluster, you must use an external database for storing state information. Many NiFi Ingest processors need to store state information. For example, Knowledge Discovery Connectors store information about what they have retrieved from a data repository. This information needs to be in an external database so that it is accessible to all of the nodes in the cluster. Configure the connection to your database server by creating a database service (see Create a Database Service). When you configure the connectors in your data flow, set the property State Database Service to the name of the database service that you created.

The files that your connectors download from your data repositories must also be accessible to all of the nodes in a cluster. When you configure your connectors, set the property `adv:IngestSharedPath` to a location, such as a shared folder, that is accessible from all of the nodes in the cluster. Alternatively, set the property `adv:FlowFileEmbedFiles` to `TRUE`, so that the binary file content is included in the FlowFiles created by the connector. For more information about these properties, see Advanced Connector Properties.

Copyright 2018-2025 Open Text
Version 25.2 | Last updated April 2025

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: Knowledge Discovery NiFi Ingest 25.2

Topic Title: Changes to your DataFlow