maxConcurrentPartitionMovesPerNode

The maxConcurrentPartitionMovesPerNode setting offers a way to speed up the rebalance operation to move partitions concurrently.

During the rebalance operations, FTS moves or builds partitions one at a time per node. It offers a way to speed up the rebalance operation to move partitions concurrently.

So, the way to speed up the rebalance operation is to enable the movement of partitions parallelly.

Setting the maxConcurrentPartitionMovesPerNode to N as a runtime cluster option can concurrently build the N number of partitions in parallel per node at a time and help in completing the rebalancing faster.

Setting maxFeedsPerDCPAgent to 1 with a sufficient FTS memory quota can help to maximize the rebalancing throughput.

Example

Speed Up the Rebalance operation:

Set maxConcurrentPartitionMovesPerNode to any number, for example 10, to bring additional concurrency:

curl -XPUT -H "Content-type:application/json" \
http://<username>:<password>@<ip>:8094/api/managerOptions \
-d '{"maxConcurrentPartitionMovesPerNode": "10"}'

Set maxConcurrentPartitionMovesPerNode to N as runtime cluster option:

curl -XPUT -H "Content-type:application/json" \
http://<username>:<password>@<ip>:8094/api/managerOptions \
-d '{"maxConcurrentPartitionMovesPerNode": "N"}'

Set maxConcurrentPartitionMovesPerNode to 1 for maximum concurrency:

curl -XPUT -H "Content-type:application/json" \
http://<username>:<password>@<ip>:8094/api/managerOptions \
-d '{"maxFeedsPerDCPAgent": "1"}'

Ensure that you have enough memory quota.