Index Statistics API

Overview

The Index Statistics REST API is provided by the Index service. This API enables you to get Index service statistics.

The API schemes and host URLs are as follows:

  • http://node:9102/

  • https://node:19102/ (for secure access)

where node is the host name or IP address of a computer running the index service.

Version information

Version : 1.0

Produces

  • application/json

Paths

Table of Contents

Get Statistics for an Index Node

GET /api/v1/stats

Description

Returns statistics for an index node, and for all indexes on that node.

Parameters

Type Name Description Schema Default

Query

pretty
optional

Whether the output should be formatted with indentations and newlines.

boolean

"false"

Query

skipEmpty
optional

Whether empty, null, or zero statistics should be omitted from the output.

boolean

"false"

Responses

HTTP Code Description Schema

200

Success. Returns an object containing a nested indexer object, and usually one or more nested keyspace:index objects — one for each index found on the specified node.

Node and Indexes

Name Description Schema

indexer
required

A nested object containing statistics for the current index node.

keyspace:index
optional

A nested object containing statistics for an entire index.

  • keyspace is the name of the keyspace.

  • index is the name of the index.

Security

Type Name

basic

Example HTTP request

Request 1: Return statistics for an index node and format the output.

Curl request
$ curl -X GET -u Administrator:password "http://localhost:9102/api/v1/stats?pretty=true"

Request 2: Return statistics for an index node, omit empty results, and format the output.

Curl request
$ curl -X GET -u Administrator:password "http://localhost:9102/api/v1/stats?skipEmpty=true&pretty=true"

Example HTTP response

Result of request 1.

Response 200
{
   "indexer": {
      "indexer_state": "Active",
      "memory_quota": 268435456,
      "memory_total_storage": 24596480,
      "memory_used": 167249920
   },
   "travel-sample:def_city": {
      "avg_drain_rate": 1,
      "avg_item_size": 59,
      "avg_scan_latency": 2581788,
      "cache_hit_percent": 100,
      "cache_hits": 7403,
      "cache_misses": 0,
      "data_size": 358088,
      "disk_size": 8192,
      "frag_percent": 0,
      "initial_build_progress": 100,
      "items_count": 7380,
      "last_known_scan_time": 1575433725539214238,
      "num_docs_indexed": 31591,
      "num_docs_pending": 0,
      "num_docs_queued": 0,
      "num_items_flushed": 31591,
      "num_pending_requests": 0,
      "num_requests": 1,
      "num_rows_returned": 7341,
      "num_scan_errors": 0,
      "num_scan_timeouts": 0,
      "recs_in_mem": 9945,
      "recs_on_disk": 0,
      "resident_percent": 100,
      "scan_bytes_read": 202869,
      "total_scan_duration": 5163576
   },
   "travel-sample:def_sourceairport_partn": {
      "avg_drain_rate": 8,
      "avg_item_size": 41,
      "avg_scan_latency": 0,
      "cache_hit_percent": 100,
      "cache_hits": 12003,
      "cache_misses": 0,
      "data_size": 1720108,
      "disk_size": 1185120,
      "frag_percent": 56,
      "initial_build_progress": 100,
      "items_count": 12003,
      "last_known_scan_time": 0,
      "num_docs_indexed": 15778,
      "num_docs_pending": 0,
      "num_docs_queued": 0,
      "num_items_flushed": 15778,
      "num_pending_requests": 0,
      "num_requests": 0,
      "num_rows_returned": 0,
      "num_scan_errors": 0,
      "num_scan_timeouts": 0,
      "recs_in_mem": 15815,
      "recs_on_disk": 0,
      "resident_percent": 100,
      "scan_bytes_read": 0,
      "total_scan_duration": 0
   }
}

Result of request 2.

Response 200
{
   "indexer": {
      "indexer_state": "Active",
      "memory_quota": 268435456,
      "memory_total_storage": 24637440,
      "memory_used": 151635968
   },
   "travel-sample:def_city": {
      "avg_item_size": 59,
      "avg_scan_latency": 2581788,
      "cache_hit_percent": 100,
      "cache_hits": 7403,
      "data_size": 859390,
      "disk_size": 618842,
      "frag_percent": 61,
      "initial_build_progress": 100,
      "items_count": 7380,
      "last_known_scan_time": 1575433725539214238,
      "num_docs_indexed": 31591,
      "num_items_flushed": 31591,
      "num_requests": 1,
      "num_rows_returned": 7341,
      "recs_in_mem": 9945,
      "resident_percent": 100,
      "scan_bytes_read": 202869,
      "total_scan_duration": 5163576
   },
   "travel-sample:def_sourceairport_partn": {
      "avg_item_size": 41,
      "cache_hit_percent": 100,
      "cache_hits": 12003,
      "data_size": 2495580,
      "disk_size": 2102624,
      "frag_percent": 64,
      "initial_build_progress": 100,
      "items_count": 12003,
      "num_docs_indexed": 15778,
      "num_items_flushed": 15778,
      "recs_in_mem": 15815,
      "resident_percent": 100
   }
}

Get Statistics for an Index

GET /api/v1/stats/{keyspace}/{index}

Description

Returns statistics for an index.

Parameters

Type Name Description Schema Default

Path

keyspace
required

The name of a keyspace.

string

Path

index
required

The name of an index.

string

Query

pretty
optional

Whether the output should be formatted with indentations and newlines.

boolean

"false"

Query

partition
optional

Whether statistics for index partitions should be included.

boolean

"false"

Query

skipEmpty
optional

Whether empty, null, or zero statistics should be omitted from the output.

boolean

"false"

It is not possible to specify an individual index partition in the path.

Responses

HTTP Code Description Schema

200

Success. Returns an object containing one nested keyspace:index object.

If the partition query parameter was set to true, the returned object also contains one or more Partition-num objects — one for each index partition found on the specified node.

404

Not found. Returns the path with the specified keyspace and index name.

The keyspace name may be incorrect, the index may not be located on the specified node, or the index may be warming up after a restart.

string

Index and Partitions

Name Description Schema

keyspace:index
required

A nested object containing statistics for an entire index.

  • keyspace is the name of the keyspace.

  • index is the name of the index.

Partition-num
optional

A nested object containing statistics.

  • If the index is partitioned, this object contains statistics for one index partition, where num is the partition number.

  • If the index is not partitioned, this object contains statistics for the entire index, and num is 0.

Security

Type Name

basic

Example HTTP request

Request 3: Return statistics for an index and format the output.

Curl request
$ curl -X GET -u Administrator:password "http://localhost:9102/api/v1/stats/travel-sample/over1000?pretty=true"

Request 4: Return statistics for an index, include partitions, and format the output.

Curl request
$ curl -X GET -u Administrator:password "http://localhost:9102/api/v1/stats/travel-sample/over1000?partition=true&pretty=true"

Example HTTP response

Result of request 3.

Response 200
{
   "travel-sample:def_sourceairport_partn": {
      "avg_drain_rate": 0,
      "avg_item_size": 41,
      "avg_scan_latency": 0,
      "cache_hit_percent": 100,
      "cache_hits": 12003,
      "cache_misses": 0,
      "data_size": 2495580,
      "disk_size": 2102624,
      "frag_percent": 64,
      "initial_build_progress": 100,
      "items_count": 12003,
      "last_known_scan_time": 0,
      "num_docs_indexed": 15778,
      "num_docs_pending": 0,
      "num_docs_queued": 0,
      "num_items_flushed": 15778,
      "num_pending_requests": 0,
      "num_requests": 0,
      "num_rows_returned": 0,
      "num_scan_errors": 0,
      "num_scan_timeouts": 0,
      "recs_in_mem": 15815,
      "recs_on_disk": 0,
      "resident_percent": 100,
      "scan_bytes_read": 0,
      "total_scan_duration": 0
   }
}

Result of request 4.

Response 200
{
   "Partition-2": {
      "avg_drain_rate": 0,
      "avg_item_size": 41,
      "avg_scan_latency": 0,
      "cache_hit_percent": 100,
      "cache_hits": 3006,
      "cache_misses": 0,
      "data_size": 625087,
      "disk_size": 528728,
      "frag_percent": 65,
      "initial_build_progress": 0,
      "items_count": 3006,
      "last_known_scan_time": 0,
      "num_docs_indexed": 3926,
      "num_docs_pending": 0,
      "num_docs_queued": 0,
      "num_items_flushed": 3926,
      "num_pending_requests": 0,
      "num_requests": 0,
      "num_rows_returned": 0,
      "num_scan_errors": 0,
      "num_scan_timeouts": 0,
      "recs_in_mem": 4010,
      "recs_on_disk": 0,
      "resident_percent": 100,
      "scan_bytes_read": 0,
      "total_scan_duration": 0
   },
   "Partition-3": {
      "avg_drain_rate": 0,
      "avg_item_size": 41,
      "avg_scan_latency": 0,
      "cache_hit_percent": 100,
      "cache_hits": 2992,
      "cache_misses": 0,
      "data_size": 622348,
      "disk_size": 520536,
      "frag_percent": 64,
      "initial_build_progress": 0,
      "items_count": 2992,
      "last_known_scan_time": 0,
      "num_docs_indexed": 3933,
      "num_docs_pending": 0,
      "num_docs_queued": 0,
      "num_items_flushed": 3933,
      "num_pending_requests": 0,
      "num_requests": 0,
      "num_rows_returned": 0,
      "num_scan_errors": 0,
      "num_scan_timeouts": 0,
      "recs_in_mem": 3996,
      "recs_on_disk": 0,
      "resident_percent": 100,
      "scan_bytes_read": 0,
      "total_scan_duration": 0
   },
   "Partition-4": {
      "avg_drain_rate": 0,
      "avg_item_size": 41,
      "avg_scan_latency": 0,
      "cache_hit_percent": 100,
      "cache_hits": 3008,
      "cache_misses": 0,
      "data_size": 625267,
      "disk_size": 528728,
      "frag_percent": 65,
      "initial_build_progress": 0,
      "items_count": 3008,
      "last_known_scan_time": 0,
      "num_docs_indexed": 3965,
      "num_docs_pending": 0,
      "num_docs_queued": 0,
      "num_items_flushed": 3965,
      "num_pending_requests": 0,
      "num_requests": 0,
      "num_rows_returned": 0,
      "num_scan_errors": 0,
      "num_scan_timeouts": 0,
      "recs_in_mem": 4011,
      "recs_on_disk": 0,
      "resident_percent": 100,
      "scan_bytes_read": 0,
      "total_scan_duration": 0
   },
   "Partition-5": {
      "avg_drain_rate": 0,
      "avg_item_size": 41,
      "avg_scan_latency": 0,
      "cache_hit_percent": 100,
      "cache_hits": 2997,
      "cache_misses": 0,
      "data_size": 622878,
      "disk_size": 524632,
      "frag_percent": 64,
      "initial_build_progress": 0,
      "items_count": 2997,
      "last_known_scan_time": 0,
      "num_docs_indexed": 3954,
      "num_docs_pending": 0,
      "num_docs_queued": 0,
      "num_items_flushed": 3954,
      "num_pending_requests": 0,
      "num_requests": 0,
      "num_rows_returned": 0,
      "num_scan_errors": 0,
      "num_scan_timeouts": 0,
      "recs_in_mem": 3798,
      "recs_on_disk": 0,
      "resident_percent": 100,
      "scan_bytes_read": 0,
      "total_scan_duration": 0
   },
   "travel-sample:def_sourceairport_partn": {
      "avg_drain_rate": 0,
      "avg_item_size": 41,
      "avg_scan_latency": 0,
      "cache_hit_percent": 100,
      "cache_hits": 12003,
      "cache_misses": 0,
      "data_size": 2495580,
      "disk_size": 2102624,
      "frag_percent": 64,
      "initial_build_progress": 100,
      "items_count": 12003,
      "last_known_scan_time": 0,
      "num_docs_indexed": 15778,
      "num_docs_pending": 0,
      "num_docs_queued": 0,
      "num_items_flushed": 15778,
      "num_pending_requests": 0,
      "num_requests": 0,
      "num_rows_returned": 0,
      "num_scan_errors": 0,
      "num_scan_timeouts": 0,
      "recs_in_mem": 15815,
      "recs_on_disk": 0,
      "resident_percent": 100,
      "scan_bytes_read": 0,
      "total_scan_duration": 0
   }
}

Definitions

Table of Contents

Node

Name Description Schema

indexer_state
optional

Current state of the Index service on this node.
Example : "Active"

enum (Active, Pause, Warmup)

memory_quota
optional

Memory quota assigned to the Index service on this node by user configuration (bytes).
Maximum value : 1099511992567
Example : 536870912

integer

memory_total_storage
optional

The total size allocated in the indexer across all indexes (bytes). This also accounts for memory fragmentation.
Example : 71794688

integer

memory_used
optional

Amount of memory used by the Index service on this node (bytes).
Example : 360192000

integer

Index

Name Description Schema

avg_array_length
optional

(Array indexes only.) The average number of items indexed per document.

integer

avg_drain_rate
optional

Average number of items flushed from memory to disk storage per second.

integer

avg_scan_latency
optional

Average time to serve a scan request (nanoseconds).

integer

cache_hit_percent
optional

Percentage of memory accesses that were served from the managed cache.

integer

cache_hits
optional

Accesses to this index data from RAM.

integer

cache_misses
optional

Accesses to this index data from disk.

integer

data_size
optional

The size of indexable data that is maintained for the index or partition (bytes).
Example : 95728

integer

docid_count
optional

(Array indexes only.) The number of documents currently indexed.

integer

disk_size
optional

Total disk file size consumed by the index or partition.
Example : 889054

integer

frag_percent
optional

Percentage fragmentation of the index.

At small index sizes of less than a hundred kB, the static overhead of the index disk file will inflate the index fragmentation percentage.

integer

initial_build_progress
optional

Percentage initial build progress for the index. When initial build is completed, the value is 100.

For an index partition, the value is listed as 0.
Example : 100

integer

items_count
optional

The number of items currently indexed.
Example : 2155

integer

last_known_scan_time
optional

Time of the last scan request received for this index (Unix timestamp in nanoseconds). This may be useful for determining whether this index is currently unused.

This statistic is persisted to disk every 15 minutes, so it is preserved when the indexer restarts.

integer

num_docs_indexed
optional

Number of documents indexed by the indexer since last startup.

integer

num_docs_pending
optional

Number of documents pending to be indexed.

integer

num_docs_queued
optional

Number of documents queued to be indexed.

integer

num_items_flushed
optional

Number of items flushed from memory to disk storage.

integer

num_pending_requests
optional

Number of requests received but not yet served by the indexer.

integer

num_requests
optional

Number of requests served by the indexer since last startup.

integer

num_rows_returned
optional

Total number of rows returned so far by the indexer.

integer

num_scan_errors
optional

Number of requests that failed due to errors other than timeout.

integer

num_scan_timeouts
optional

Number of requests that timed out, either waiting for snapshots or during scan in progress.

integer

recs_in_mem
optional

For standard index storage, this is the number of records in this index that are stored in memory.

For memory-optimized index storage, this is the same as items_count.
Example : 2155

integer

recs_on_disk
optional

For standard index storage, this is the number of records in this index that are stored on disk.

For memory-optimized index storage, this is 0.

integer

resident_percent
optional

Percentage of the data held in memory.
Example : 100

integer

scan_bytes_read
optional

Number of bytes read by a scan since last startup.

integer

total_scan_duration
optional

Total time spent by the indexer in scanning rows since last startup.

integer

Security

Default

The Index Statistics API supports admin credentials. Credentials can be passed via HTTP headers (HTTP basic authentication).

Users must have Cluster Read and Bucket INDEX List privileges. Refer to Roles for more details.

Type : basic