Whenever Couchbase Server is restarted, or data is restored to a server instance, the server undergoes a warmup process before data requests can be handled.
During warmup, Couchbase Server loads data persisted on disk into RAM.
Couchbase Server provides an optimized warmup process that loads data sequentially from disk into RAM. It divides the data to be loaded and handles it in multiple phases. After the warmup process completes, the data is available for clients to read and write. The time needed for server warmup depends on system size, system configuration and the amount of data persisted in the system.
Couchbase Server identifies items that are frequently used, prioritizes them, and loads them before sequentially loading the remaining data. The frequently-used items are prioritized in an access log. The server pre-fetches a list of the most frequently accessed keys and then fetches these documents before any other items from disk.
The server runs a configurable scanner process that determines which keys are most frequently used. The scanner process is pre-set and is configurable. The command-line tool, cbepctl flush_param, is used to change the initial time and interval for the scanner process. For example, you might want to configure the scanner process to be run during a specific time period when certain keys need to be identified and made available sooner.
The server can also switch into a ready mode before it has actually retrieved all documents for keys into RAM. Therefore, data can be served before all stored items are loaded. Switching into ready mode is a configurable setting so that server warmup time can be adjusted.
In these first phase of the initial warmup, Couchbase Server begins to fetch all keys and metadata from disk. Afterward, it accesses the log information that it needs to retrieve the most used keys:
- 1. Initialize
- In this phase, Couchbase Server has no data it can serve. It starts populating a list of all vBuckets stored on disk by loading the recorded, initial state of each vBucket.
- 2. Key dump
- Couchbase Server starts pre-fetching all keys and metadata from disk based on the items in the vBucket list.
- 3. Check access log
- Couchbase Server reads a single cached access log that indicates which keys are frequently accessed. It generates and maintains this configurable log on a periodic basis. If this log exists, the server first loads items based on it before it loads other items from disk.
Once Couchbase Server has the information about keys and has read in any access log information, it loads documents based on the following criteria:
- Loading based on access logs
- Couchbase Server loads documents into memory based on the frequently-used items identified in the access log.
- Loading data
- If the access log is empty or is disabled, Couchbase Server sequentially loads documents for each key based on the vBucket list.
Couchbase Server is able to serve information from RAM when one of the following conditions is met during warmup:
- The server has finished loading documents for all keys listed in the access log.
- The server has finished loading documents for every key stored on disk for all vBuckets.
- The percentage of documents loaded into memory is greater than, or equal to, the setting for the cbepctl ep_warmup_min_items_threshold parameter.
- Total % of RAM filled by documents is greater than, or equal to, the setting for the cbepctl ep_warmup_min_memory_threshold parameter
- Total RAM usage by a node is greater than or equal to the setting for mem_low_wat.
When Couchbase Server reaches one of these states, known as the run level, it stops loading documents for the remaining keys. There isn't any further loading of data unless the document is requested by a client, triggering a background fetch.