Couchbase C++ SDK 1.5.1 (rev. 33c90f2)
Loading...
Searching...
No Matches
cbc-pillowfight - Simple Workload Generator

NAME

cbc pillowfight - run workload generator

SYNOPSIS

cbc pillowfight [options] <id>...
cbc pillowfight (-h|--help)

DESCRIPTION

Run simple workload generator that sends GET/UPSERT requests with optional N1QL queries.

OPTIONS

-h|--help
Show this screen.
--verbose
Include more context and information where it is applicable.
--bucket-name=STRING
Name of the bucket. [default: default]
--scope-name=STRING
Name of the scope. [default: _default]
--collection-name=STRING
Name of the collection. [default: _default]
--operation-batch-size=INTEGER
Number of the operations in a single batch (set to 1 to wait for completion after every operation). [default: 100]
--batch-wait=DURATION
Time to wait after the batch. [default: 0ms]
--number-of-io-threads=INTEGER
Number of the IO threads. [default: 1]
--number-of-worker-threads=INTEGER
Number of the IO threads. [default: 1]
--operation-ratio=TEXT
The ratio of the operations to generate in form "G:R:D:I:Q", where letters represent ratio of the operations in whole numbers: Get, Replace, Delete, Insert and Query respectively. (e.g. "5:0:0:1:0" would do on average 5 gets for every insert). [default: 1:1:0:0:0]
--query-statement=STRING
The N1QL query statement to use ({bucket_name}, {scope_name} and {collection_name} will be substituted). [default: SELECT COUNT(*) FROM `{bucket_name}` WHERE type = "fake_profile"]
--document-body-size=INTEGER
Size of the body (if zero, it will use predefined document). [default: 0]
--document-body-fill=MODE
How to fill a generated document body (allowed values: constant, random). constant repeats a single character; random draws every document from a seeded pseudo-random generator. Only random gives each document distinct bytes. [default: constant]
--document-body-format=FORMAT
Format of a generated document (allowed values: json, binary). json wraps the body in a JSON object and restricts it to a JSON-safe alphabet; binary stores the body as opaque bytes of exactly --document-body-size. [default: json]
--document-body-pool-size=INTEGER
Pre-generate this many bytes of document bodies per worker thread and cycle through them, instead of generating a fresh body for every document. Measured in bytes, with a floor of 1 MiB. [default: 0]
--document-body-seed=INTEGER
Seed for --document-body-fill=random, so that a dataset can be reproduced. Each worker thread offsets it by its index. Without it the generator is seeded from the system entropy source.
--number-of-keys-to-populate=INTEGER
Preload keys before running workload, so that the worker will not generate new keys afterwards. [default: 1000]
--operations-limit=INTEGER
Stop and exit after the number of the operations reaches this limit. (zero for running indefinitely) [default: 0]

LOADING INCOMPRESSIBLE DOCUMENTS

Sizing a dataset to a target disk footprint needs documents that do not compress, so that the bytes written follow from the number of documents. Two compressors stand in the way, and they work at different scopes.

Per document, the SDK and the Key/Value service compress with snappy, keeping the compressed form only when it is smaller by a configured margin — --compression-minimum-ratio here, min_compression_ratio on the server. Random bytes never clear that bar.

Per data block, the storage engine compresses several documents together, with lz4 by default. This is the one that matters: a body reused across documents is stored once and back-referenced for the rest of the block, so a load of a million identical documents occupies a fraction of their combined size no matter how random that single body looks. --document-body-fill=random gives every document its own bytes.

Measured over the generator's own output, compressed in 4096-byte blocks:

--document-body-fill / --document-body-format lz4 snappy zstd
constant / json 67.79x 18.29x 80.72x
random / json 1.00x 1.00x 1.33x
random / binary 1.00x 1.00x 1.00x

A JSON body has to stay inside a JSON string, so it is drawn from a 62-character alphabet. That is invisible to lz4 and snappy, neither of which entropy-codes, but not to zstd, which magma_data_compression_algo can be set to. Binary bodies are drawn from the full byte range and hold at 1.00x whichever algorithm is configured.

To load 100 million documents of exactly 1024 bytes:

cbc pillowfight --bucket-name default \
--document-body-size 1024 \
--document-body-fill random \
--document-body-format binary \
--number-of-keys-to-populate 100000000 \
--operations-limit 1

--operations-limit 1 ends the workload phase at the first completed batch, so the run is the load and little else. --document-body-format=binary makes --document-body-size the exact size of the stored document; under json the body is wrapped in an object and the document is larger than the value given. Binary documents are opaque to the Query service, so --operation-ratio should not ask for queries alongside it.

--document-body-pool-size bounds the work spent per document by pre-generating bodies and cycling them. It is rarely worth setting: generating a fresh body costs a fraction of a microsecond, far less than the operation carrying it. When it is set, it is measured in bytes, not documents, because what has to be defeated is the compressor's look-back distance — a distance in bytes, which a document count does not describe. A pool smaller than that distance puts the same body into one block twice and the documents compress again, silently and by a large factor. That is why the option refuses a pool below 1 MiB rather than warning about it.

LOGGER OPTIONS

--log-level=LEVEL
Log level (allowed values are: trace, debug, info, warning, error, critical, off). [default: off]
--log-output=PATH
File to send logs (when is not set, logs will be written to STDERR).
--log-protocol=PATH
File to send protocol logs.

CONNECTION OPTIONS

--connection-string=STRING
Connection string for the cluster. [default: couchbase://localhost]
--username=STRING
Username for the cluster. [default: Administrator]
--password=STRING
Password for the cluster. [default: password]
--certificate-path=STRING
Path to the certificate.
--key-path=STRING
Path to the key.
--ldap-compatible
Whether to select authentication mechanism that is compatible with LDAP.
--configuration-profile=STRING
Apply configuration profile (might override other switches). (available profiles: wan_development)

SECURITY OPTIONS

--disable-tls
Whether to disable TLS.
--trust-certificate-path=STRING
Path to the trust certificate bundle.
--tls-verify-mode=MODE
Path to the certificate (allowed values: peer, none). [default: peer]

TIMEOUT OPTIONS

--bootstrap-timeout=DURATION
Timeout for overall bootstrap of the SDK. [default: 10000ms]
--connect-timeout=DURATION
Timeout for socket connection. [default: 10000ms]
--resolve-timeout=DURATION
Timeout to resolve DNS address for the sockets. [default: 2000ms]
--key-value-timeout=DURATION
Timeout for Key/Value operations. [default: 2500ms]
--key-value-durable-timeout=DURATION
Timeout for Key/Value durable operations. [default: 10000ms]
--query-timeout=DURATION
Timeout for Query service. [default: 75000ms]
--search-timeout=DURATION
Timeout for Search service. [default: 75000ms]
--eventing-timeout=DURATION
Timeout for Eventing service. [default: 75000ms]
--analytics-timeout=DURATION
Timeout for Analytics service. [default: 75000ms]
--view-timeout=DURATION
Timeout for View service. [default: 75000ms]
--management-timeout=DURATION
Timeout for management operations. [default: 75000ms]

COMPRESSION OPTIONS

--disable-compression
Whether to disable compression.
--compression-minimum-size=INTEGER
The minimum size of the document (in bytes), that will be compressed. [default: 32]
--compression-minimum-ratio=FLOAT
The minimum compression ratio to allow compressed form to be used. [default: 0.83]

DNS-SRV OPTIONS

--dns-srv-timeout=DURATION
Timeout for DNS SRV requests. [default: 500ms]
--dns-srv-nameserver=STRING
Hostname of the DNS server where the DNS SRV requests will be sent.
--dns-srv-port=INTEGER
Port of the DNS server where the DNS SRV requests will be sent.

NETWORK OPTIONS

--tcp-keep-alive-interval=DURATION
Interval for TCP keep alive. [default: 60000ms]
--config-poll-interval=DURATION
How often the library should poll for new configuration. [default: 2500ms]
--idle-http-connection-timeout=DURATION
Period to wait before calling HTTP connection idle. [default: 4500ms]

TRANSACTIONS OPTIONS

--transactions-durability-level=LEVEL
Durability level of the transaction (allowed values: none, majority, majority_and_persist_to_active, persist_to_majority). [default: majority]
--transactions-timeout=DURATION
Timeout of the transaction. [default: 15000ms]
--transactions-metadata-bucket=STRING
Bucket name where transaction metadata is stored.
--transactions-metadata-scope=STRING
Scope name where transaction metadata is stored. [default: _default]
--transactions-metadata-collection=STRING
Collection name where transaction metadata is stored. [default: _default]
--transactions-query-scan-consistency=MODE
Scan consistency for queries in transactions (allowed values: not_bounded, request_plus). [default: request_plus]
--transactions-cleanup-ignore-lost-attempts
Do not cleanup lost attempts.
--transactions-cleanup-ignore-client-attempts
Do not cleanup client attempts.
--transactions-cleanup-window=DURATION
Cleanup window. [default: 60000ms]

METRICS OPTIONS

--disable-metrics
Disable collecting and reporting metrics.
--metrics-emit-interval=DURATION
Interval to emit metrics report on INFO log level. [default: 600000ms]

TRACING OPTIONS

--disable-tracing
Disable collecting and reporting trace information.
--tracing-orphaned-emit-interval=DURATION
Interval to emit report about orphan operations. [default: 10000ms]
--tracing-orphaned-sample-size=INTEGER
Size of the sample of the orphan report. [default: 64]
--tracing-threshold-emit-interval=DURATION
Interval to emit report about operations exceeding threshold. [default: 10000ms]
--tracing-threshold-sample-size=INTEGER
Size of the sample of the threshold report. [default: 64]
--tracing-threshold-key-value=DURATION
Threshold for Key/Value service. [default: 500ms]
--tracing-threshold-query=DURATION
Threshold for Query service. [default: 1000ms]
--tracing-threshold-search=DURATION
Threshold for Search service. [default: 1000ms]
--tracing-threshold-analytics=DURATION
Threshold for Analytics service. [default: 1000ms]
--tracing-threshold-management=DURATION
Threshold for Management operations. [default: 1000ms]
--tracing-threshold-eventing=DURATION
Threshold for Eventing service. [default: 1000ms]
--tracing-threshold-view=DURATION
Threshold for View service. [default: 1000ms]

BEHAVIOR OPTIONS

--user-agent-extra=STRING
Append extra string SDK identifiers. [default: cbc].
--network=STRING
Network (a.k.a. Alternate Addresses) to use. [default: auto]
--show-queries
Log queries on INFO level.
--enable-clustermap-notifications
Allow server to send notifications when cluster configuration changes.
--disable-mutation-tokens
Do not request Key/Value service to send mutation tokens.
--disable-unordered-execution
Disable unordered execution for Key/Value service.

ENVIRONMENT

CBC_LOG_LEVEL
Overrides default value for --log-level.
CBC_CONNECTION_STRING
Overrides default value for --connection-string.
CBC_USERNAME
Overrides default value for --username.
CBC_PASSWORD
Overrides default value for --password.

SEE ALSO

cbccbc.