maxConcurrentPartitionMovesPerNode
The maxConcurrentPartitionMovesPerNode
setting offers a way to speed up the rebalance operation to move partitions concurrently.
During the rebalance operations, FTS moves or builds partitions one at a time per node. It offers a way to speed up the rebalance operation to move partitions concurrently.
So, the way to speed up the rebalance operation is to enable the movement of partitions parallelly.
Setting the maxConcurrentPartitionMovesPerNode
to N as a runtime cluster option can concurrently build the N number of partitions in parallel per node at a time and help in completing the rebalancing faster.
Setting maxFeedsPerDCPAgent
to 1 with a sufficient FTS memory quota can help to maximize the rebalancing throughput.
Example
Speed Up the Rebalance operation:
-
Set
maxConcurrentPartitionMovesPerNode
to any number, for example 10, to bring additional concurrency:curl -XPUT -H "Content-type:application/json" http://<username>:<password>@<ip>:8094/api/managerOptions \-d '{"maxConcurrentPartitionMovesPerNode": "10"}
-
Set
maxConcurrentPartitionMovesPerNode
to N as runtime cluster option:curl -XPUT -H "Content-type:application/json" http://<username>:<password>@<ip>:8094/api/managerOptions \-d '{"maxConcurrentPartitionMovesPerNode": "N"}'
-
Set
maxConcurrentPartitionMovesPerNode
to 1 for maximum concurrency:curl -XPUT -H "Content-type:application/json" http://<username>:<password>@<ip>:8094/api/managerOptions \-d '{"maxFeedsPerDCPAgent": "1"}'
Ensure that you have enough memory quota.