A newer version of this documentation is available.

View Latest

set flush_param

  • reference
    +
    The command set flush_param establishes bucket parameters for threading and memory management.

    Syntax

    The basic syntax is:

    cbepctl [host]:11210 -b [bucket-name] -u [administrator-name] -p [administrator-password] set flush_param [parameter] [value]

    The syntax to configure the access log settings is:

    cbepctl [hostname]:11210 -b [bucket-name] -u [administrator-name] -p [administrator-password] set flush_param alog_sleep_time [value]
    cbepctl [hostname]:11210 -b [bucket-name] -u [administrator-name] -p [administrator-password] set flush_param alog_task_time [value]

    The syntax for disk cleanup is:

    cbepctl [host]:11210 -b [bucket-name] -u [administrator-name] -p [administrator-password] set flush_param exp_pager_stime [value]

    The syntax for ejection is:

    cbepctl [host]:11210 -b [bucket-name] -u [administrator-name] -p [administrator-password] set flush_param [parameter] [value]

    Parameters used for changing ejection thresholds:

    • mem_low_wat

    • mem_high_wat

    • pager_active_vb_pcnt

    The syntax to set the out of memory threshold is:

    cbepctl [host]:11210 -b [bucket-name] -u [administrator-name] -p [administrator-password] set flush_param mutation_mem_threshold [value]

    Description

    Tune the dynamic shared thread pool performance by changing the thread types inside the ep-engine and memcached at run time. The command set flush_param adjusts the number of threads that prioritize read, write, non-i/o and auxiliary-i/o operations. These settings take effect immediately and do not require that the bucket be restarted.

    The settings for threads number take effect only if the underlying operating system has a sufficient number of CPU cores. The minimum number of CPU cores is four (4), but three (3) additional cores are required for each additional writer thread. For example, five (5) writer threads is a valid setting if the underlying hardware has at least sixteen (16) cores.
    Changes of thresholds are NOT persistent and must be reapplied after the bucket warmup.
    alog_sleep_time, alog_task_time

    Couchbase Server has an optimized disk warmup. An access scanner is periodically run, to determine which keys have been the most frequently used: typically, the scanner writes these key to an access log, which can be accessed by Couchbase Server at warmup, so that the corresponding documents can be loaded first. (See Initialization and Warmup for further information; including cases where item-residency is at very high levels, and in consequence, no access log is created or used.)

    The cbepctl flush_param command is used to change the initial time and the interval for the access scanner. For example, the initial time and interval might be changed to accommodate a peak time when an application needs these keys to be quickly available.

    By default, the access scanner runs once every 24 hours at 10:00 AM GMT. The scanner is highly CPU-intensive: therefore, to reduce the cluster-wide impact of running this task, its start time should be staggered to a different value on each node in the cluster. Note also that if the scanner runs at the same time that index updates are being made (either on the current node, or on one or more other nodes) by the Index Service, the performance of the index updates may be adversely affected. The scanner should be configured to minimize the likelihood of this problem.

    exp_pager_stime

    The cbepctl flush_param exp_pager_stime command sets the time interval for disk cleanup. Couchbase Server does lazy expiration, that is, expired items are flagged as deleted rather than being immediately erased. Couchbase Server has a maintenance process that periodically looks through all information and erases expired items. By default, this maintenance process runs every 10 minutes, but it can be configured to run at a different interval.

    The compaction process will also remove expired items.
    mem_low_wat, mem_high_wat, pager_active_vb_pcnt

    Ejection means that documents are removed from RAM but the key and metadata remain. If the amount of RAM used by items reaches the high water mark (upper threshold), both active and replica data are ejected until the memory usage (amount of RAM consumed) reaches the low water mark (lower threshold). Ejected items are those that have not been recently used.

    Use the mem_low_wat, mem_high_wat, and pager_active_vb_pcnt settings to change the server thresholds for ejection.

    Do not change the ejection defaults unless required by Couchbase Support.
    mutation_mem_threshold

    By default, Couchbase Server sends clients a temporary out-of-memory error message if RAM is 95% consumed and only 5% RAM remains for overhead. Use the cbepctl set flush_param mutation_mem-threshold command parameter to change this threshold value.

    Do not change this default to a higher value. However, this value might be reduced if you need more RAM for system overhead such as disk queue or for server data structures.

    Options

    The following are the command options:

    Table 1. set flush_param options
    Option Description

    alog_sleep_time

    Access scanner interval (minute)

    alog_task_time

    Access scanner next task time (UTC)

    backfile_mem_threshold

    Memory threshold (%) on the current bucket quota before backfill task is made to back off.

    bg_fetch_delay

    Delay before executing a bg fetch (test feature).

    couch_response_timeout

    timeout in receiving a response from CouchDB.

    exp_pager_stime

    Expiry Pager interval. Time interval that Couchbase Server waits before it performs cleanup and removal of expired items from disk. Setting this value to 0 will disable the Expiry Pager from running.

    flushall_enabled

    Deprecated. Enable flush operation.

    pager_active_vb_pcnt

    Percentage of active vBuckets items among all ejected items by item pager.

    max_size

    Maximum memory used by the server.

    mem_high_wat

    High water mark in bytes.

    mem_low_wat

    Low water mark in bytes.

    mutation_mem_threshold

    Amount of RAM that can be consumed in that caching layer before clients start receiving temporary out of memory messages.

    timing_log

    Path to log detailed timing stats.

    warmup_min_memory_threshold

    Memory threshold (%) during warmup to enable traffic.

    warmup_min_items_threshold

    Item number threshold (%) during warmup to enable traffic.

    klog_compactor_queue_cap

    Queue cap to throttle the log compactor.

    klog_max_log_size

    Maximum size of a mutation log file allowed.

    klog_max_entry_ratio

    Max ratio of # of items logged to # of unique items.

    pager_unbiased_period

    Period after last access scanner run during which item pager preserve working set.

    queue_age_cap

    Maximum queue age before flushing data.

    max_txn_size

    Maximum number of items in a flusher transaction.

    min_data_age

    Minimum data age before flushing data.

    item_compressor_interval

    How often the item compressor task should be run, in milliseconds. Default value is 250.

    item_compressor_chunk_duration

    Maximum time, in milliseconds, for which the item compressor task is run, before being paused, and then resumed according to the established item_compressor_interval. Default value is 20.

    min_compression_ratio

    Minimum allowed ratio of each item’s uncompressed form to its compressed form. If the actual ratio is less than this value, the item is stored in uncompressed form. Default value is 1.2.

    % You must use the percentage sign in order to set the value by percentage.

    Examples

    Examples for setting the access scanner process

    To change the time interval when the access scanner process runs to every 20 minutes.

    cbepctl 10.5.2.117:11210 -b foo-bucket -u Administrator -p password \
    set flush_param alog_sleep_time 20

    To change the initial time that the access scanner process runs from the 2:00 AM UTC default to 11:00 PM UTC.

    cbepctl 10.5.2.117:11210 -b foo-bucket -u Administrator -p password \
    set flush_param alog_task_time 23

    This response shows the time interval changed to 20 minutes.

    setting param: alog_sleep_time 20
    set alog_sleep_time to 20

    This response shows the initial access scanner run time changed to 11:00 PM UTC.

    setting param: alog_task_time 23
    set alog_task_time to 23

    Examples for setting the disk cleanup

    The following example sets the cleanup process to run every 600 seconds (10 minutes). This is the interval that Couchbase Server waits before it tries to remove expired items from disk.

    cbepctl 10.5.2.117:11210 -b foo-bucket -u Administrator -p password \
    set flush_param exp_pager_stime 600

    The following example response shows the cleanup process set to 600 seconds.

    setting param: exp_pager_stime 600
    set exp_pager_stime to 600

    Examples for setting the out-of-memory error message

    In this example, the threshold is reduced to 65% of RAM.

    cbepctl 10.5.2.117:11210 -b foo-bucket -u Administrator -p password \
    set flush_param mutation_mem_threshold 65%

    The following example response shows the RAM threshold set to 65%.

    setting param: mutation_mem_threshold 65
    set mutation_mem_threshold to 65

    Example for setting the low water mark

    The low water mark sets the lower threshold of RAM for a specific bucket on a node. The item pager stops ejecting items once the low water mark is reached.

    The following example sets the low water mark percentage to 70% of RAM.

    cbepctl 10.5.2.117:11210 -b foo-bucket -u Administrator -p password \
    set flush_param mem_low_wat 70%

    Example for setting the high water mark

    The high water mark set the amount of RAM consumed by items that must be breached before infrequently used active and replica items are ejected.

    The following example sets the high water mark percentage to 80% of RAM for a specific bucket on a node. This means that items in RAM on this node can consume up to 80% of RAM before the item pager begins ejecting items.

    cbepctl 10.5.2.117:11210 -b foo-bucket -u Administrator -p password \
    set flush_param mem_high_wat 80%

    Examples for setting percentage of ejected items

    The server ejects active and replica data from a node, when that data has not been recently used. By default, the server is configured to 60% active items and 40% replica data from a node.

    The following example increases the percentage of active items that can be ejected from a node to 50%.

    cbepctl 10.5.2.117:11210 -b foo-bucket -u Administrator -p password \
    set flush_param pager_active_vb_pcnt 50

    Be aware of potential performance implications when changing the percentage of ejected items. It may be more desirable to eject as many replica items as possible and limit the amount of active data that can be ejected. By doing so, active data from a source node is maximized while maintaining incoming requests to that node. However, if the server is ejecting a very large percentage of replica data and a node fails, the replica data is not immediately available. In this case, the items are retrieved from disk and put back into RAM before the request is fulfilled.

    The following example response shows the low water mark, high water mark, and percentage of ejected items being set.

    setting param: mem_low_wat 70
    set mem_low_wat to 70
    
    setting param: mem_high_wat 80
    set mem_high_wat to 80
    
    setting param: pager_active_vb_pcnt 50
    set pager_active_vb_pcnt to 50