To create an Apache NiFi cluster, set up either three or five ZooKeeper nodes.
To set up ZooKeeper nodes for a NiFi cluster
Open the file ./conf/nifi.properties
and set or review the following properties:
Property | Description |
---|---|
nifi.cluster.is.node
|
Set this property to true . |
nifi.state.management.embedded.zookeeper.start
|
Specifies whether this node runs an embedded ZooKeeper server. Set this property to true . |
nifi.zookeeper.connect.string
|
A comma-separated list of host names and ports, for connecting to ZooKeeper on each node. The port number you set here must match the ZooKeeper client port that you set later (see step 4). For example:
|
nifi.web.http.port
|
The NiFi HTTP Port. This is the port that you would use to open the Apache NiFi web interface. |
nifi.cluster.node.protocol.port
|
Specify any free port above 1024. |
nifi.cluster.load.balance.port
|
Specify any free port above 1024. |
nifi.state.management.embedded.zookeeper.properties
|
This property specifies the path to the ZooKeeper configuration file. You can use the default value. |
Open the file ./conf/state-management.xml
and set the Connect String
property (in the ZooKeeper section) to the same value you set for nifi.zookeeper.connect.string
in the nifi.properties
file. For example:
<property name="Connect String">host1:4881,host2:4881,host3:4881</property>
Open the file ./conf/zookeeper.properties
.
Apache NiFi 1.10.x - List the nodes that you have created. For example:
server.1=host1:2881:3881;4881
server.2=host2:2881:3881;4881
server.3=host3:2881:3881;4881
The first two port numbers on each row are used by ZooKeeper. Specify any free port above 1024. The last port number on each row must correspond to the port you specified in the property nifi.zookeeper.connect.string
in the nifi.properties
file.
Apache NiFi 1.9.x and earlier - Complete the following steps:
Set the clientPort
parameter to the port you specified for this node in nifi.zookeeper.connect.string
in the nifi.properties
file. For example:
clientPort=4881
List the nodes that you have created. For example:
server.1=host1:2881:3881
server.2=host2:2881:3881
server.3=host3:2881:3881
These port numbers are used by ZooKeeper. Specify any free ports above 1024.
Create a text file ./state/zookeeper/myid
that contains the server index. In the previous step you edited zookeeper.properties
and specified that host1
has a server index of 1
(because it is listed as server.1
), host2
has a server index of 2
(server.2
), and host3
has a server index of 3
(server.3
). The text file that you create should contain only the number and nothing else.
Open the NiFi user interface for any node in the cluster.
|