Backfill Support for SQL++
Configure the temporary working space for the SQL++ engine and its embedded GSI client.
Introduction
When a query has an extremely large corresponding index scan, the indexer gsiClient
buffers the results into the /tmp
directory.
Since this method has caused some high I/O and works differently on Windows, Couchbase Server enables you to configure the backfill directory via the UI, which will then pass the settings to the server.
Prerequisites
The user must have Cluster Admin privileges in order to be able to specify and configure the location for temporarily storing any transient-data during query processing per query-engine.
Configuration
In the Query Workbench > Settings page, click Advanced Query Settings to see:
In the Query Temp Disk Path textbox, enter the directory you want to use for your backfill file.
In the Quota textbox, enter the number of megabytes for the maximum size of your backfill file.
Your system administrator can read and modify the two REST backfill parameters:
Setting | Description | Values | Remarks |
---|---|---|---|
|
The temp directory in which the indexer can create the |
Default value is |
Only absolute paths are allowed. |
|
The maximum limit in MB for the temporary file size. |
|
The max size is limited only by your available disk space. |
Examples
$ curl --GET -u Administrator:password http://localhost:8091/settings/querySettings
{"queryTmpSpaceDir":"/opt/couchbase/tmp","queryTmpSpaceSize":5120,"queryCurlWhitelist":{"all_access":false,"allowed_urls":["http://localhost:8091/settings/querySettings",""],"disallowed_urls":[""]}}
tmp2
(after making a tmp2
directory)$ curl -X POST -u Administrator:password -d 'queryTmpSpaceDir=/opt/couchbase/tmp2' http://localhost:8091/settings/querySettings
{"queryTmpSpaceDir":"/opt/couchbase/tmp2","queryTmpSpaceSize":5120,"queryCurlWhitelist":{"all_access":false,"allowed_urls":["http://localhost:8091/settings/querySettings",""],"disallowed_urls":[""]}}
$ curl -X POST -u Administrator:password -d 'queryTmpSpaceDir=/opt/couchbase/tmp' -d 'queryTmpSpaceSize=2048' http://localhost:8091/settings/querySettings
{"queryTmpSpaceDir":"/opt/couchbase/tmp","queryTmpSpaceSize":2048,"queryCurlWhitelist":{"all_access":false,"allowed_urls":["http://localhost:8091/settings/querySettings",""],"disallowed_urls":[""]}}
At the same time, the logs will contain messages for every change coming from both query and GSI, so after executing Example 3’s CURL statement:
_time=2017-12-06T14:12:39.709-08:00 _level=INFO _msg=New settings received: {"query.settings.tmp_space_dir":"/opt/couchbase/tmp/","query.settings.tmp_space_size":2048} _time=2017-12-06T14:12:39.712-08:00 _level=INFO _msg=Settings notifier from metakv 2017-12-06T14:12:39.715-08:00 [Info] GSIC - Setting config map[query_tmpspace_dir:/opt/couchbase/tmp/ query_tmpspace_limit:2048] _time=2017-12-06T14:12:39.715-08:00 _level=INFO _msg= Indexer settings have been updated map[query_tmpspace_dir:/opt/couchbase/tmp/ query_tmpspace_limit:2048]