NiFi Ingest 25.2
NiFi Ingest

# Optimize your Data Flow

To maximize throughput, monitor the size of the queues across your dataflow. If a large queue builds up before a processor, and the queues for the subsequent processors are empty, the processor is causing a bottleneck. This isn't unusual because some tasks are expected to take longer. To improve throughput you can configure a processor to process multiple FlowFiles concurrently.

To process multiple FlowFiles concurrently

1. Right-click the processor and click Configure.
2. Click the Scheduling tab.

3. In the Concurrent Tasks box, type the number of CPU threads to use.

   NOTE: The total number of CPU threads that you can use is limited by your hardware. If your server's CPU is under maximum load, adding additional threads does not help. Additionally, NiFi places a limit on the maximum number of threads, which you might need to increase. For more information see Maximum Number of Threads.

Sometimes, allowing a processor to use more resources just moves the bottleneck to a processor further down the pipeline. You might need to repeat this process several times and monitor the queue sizes until the CPU threads are distributed correctly.

Copyright 2018-2025 Open Text
Version 25.2 | Last updated April 2025

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: Knowledge Discovery NiFi Ingest 25.2

Topic Title: Optimize your Data Flow