Scopes and Collections

      +
      Couchbase Server provides scopes and collections; allowing documents to be categorized and organized, within a bucket.

      This page contains an in-depth look at scopes and collections in Couchbase. If you are looking for an introduction to logically partitioning your data with scopes and collections, you may like to start with the introduction to the topic in our developer tutorial.

      Understanding Scopes and Collections

      A collection is a data container, defined on Couchbase Server, within a bucket whose type is either Couchbase or Ephemeral. Up to 1000 collections can be created per cluster. Item-names must be unique within their collection. Items can optionally be assigned to different collections according to content-type. For example, within a bucket that contains travel information, documents that relate specifically to airports might be assigned to an airports collection, while documents that relate to hotels might be assigned to a hotels collection, and so on. Applications can be assigned per-collection access-rights; allowing each application to access only those collections it requires.

      A collection can be created and managed by means of the Couchbase Web Console UI, the Couchbase CLI, the Couchbase REST API, and SQL++. It can also be created and managed by means of the SDK: see Collection Management. A collection can be indexed; and it can be dropped. The data in a collection can be replicated, by means of Cross Data Center Replication (XDCR), to another collection in the same or in a different bucket. A collection is named by the administrator when created: it cannot subsequently be renamed.

      A scope is a mechanism for the grouping of multiple collections. Up to 1000 scopes can be created per cluster. Collection-names must be unique within their scope. Collections might be assigned to different scopes according to content-type, or to deployment-phase (ie, test versus production). Applications can be assigned per-scope access-rights; allowing each application to access only those scopes it requires.

      A scope can be created and managed by means of the Couchbase Web Console UI, the Couchbase CLI, the Couchbase REST API, and SQL++. It can also be created and managed by means of the SDK: see Collection Management. A scope can be dropped. A scope cannot be indexed. The contents of a scope can be replicated, by means of Cross Data Center Replication (XDCR), to another scope in the same or in a different bucket. A scope is named by the administrator when created: it cannot subsequently be renamed.

      Benefits of Scopes and Collections

      The benefits of scopes and collections include:

      • The logical grouping of similar documents; potentially simplifying operations such as query, XDCR, and backup and restore.

      • The increased efficiency of indexing, due to the Data Service being able to provide documents from specific collections to the Index Service.

      • Simplified querying, since query statements are able to easily specify particular subsets of documents.

      • Easier migration from relational databases to Couchbase Server, since collections can be designed to correspond to pre-existing relational tables.

      • Secure isolation of different document-types, within a bucket; allowing applications to be specifically authorized to use only their appropriate subsets of data (see Access to Scopes and Collections, below).

      Naming for Scopes and Collections

      User-defined scopes and collections must be assigned user-defined names. Such names:

      • Must be between 1 and 251 characters in length.

      • Can only contain the characters A-Z, a-z, 0-9, and the symbols _, -, and %. Note that scope and collection names are case sensitive.

      • Cannot start with _ or %.

      The namespace within a scope is independent of any other namespace within any other scope: consequently, the same collection-name can exist in multiple scopes, within a single bucket. Similarly, the namespace within a collection is independent of any other namespace within any other collection: consequently, the same document-key can exist in multiple collections, within a single scope.

      Default Scope and Collection

      Every created bucket is automatically given a default scope, and within it, a default collection. Each is named _default. All documents created within the bucket without reference to specific scopes or collections are saved in the default collection, within the default scope. On upgrade from a version of Couchbase Server prior to 7.0, all existing data is automatically placed in the default scope and default collection.

      The default scope cannot be dropped. The default collection can be dropped, by means of either the Couchbase CLI, or the REST API. Once dropped, the default collection cannot be recreated.

      The _system Scope, and its Collections

      In Couchbase Server Version 7.6+, in each user-created or sample bucket, a _system scope is created and maintained by default. This scope contains collections used by Couchbase services, for service-specific data. The scope and its collections cannot be dropped.

      Users should NOT rely on the content of those collections. Their structure is subject to change without warning.

      For reference, the collections are as follows:

      _mobile

      Sync-Gateway owned metadata.

      _query

      This collection stores:

      • Cost-Based Optimizer statistics, gathered through UPDATE STATISTICS/ANALYZE

      • Metadata about Query side User Defined Functions.

      Access to Scopes and Collections

      Access to scopes and collections is protected by Role-Based Access Control (RBAC): therefore, appropriate roles must be assigned.

      For information on RBAC, see Authorization. For a list of roles, see Roles. For examples of how to assign roles to scopes and collections, see Manage Users, Groups, and Roles.

      Tenant Separation

      A query may refer to a collection using an absolute keyspace path; or a relative partial keyspace reference, which must be resolved by means of the query context. The use of partial keyspace references and query context supports the separation of tenant data in a multi-tenancy environment. For details, refer to Query Context.

      Expiration and Collections

      An expiration value can be established on a collection: this determines the maximum expiration time of individual items within the collection. For information, see Expiration.

      Change History

      When Magma storage is used for a bucket, the changes made to documents within the bucket’s collections can be recorded, in a change history. The change history resides on disk. Its capacity is administrator-specified. When the change history is full, old records are automatically removed (by means of compaction), to allow space for new records.

      For detailed information, see Change History.

      Managing Scopes and Collections

      See Manage Scopes and Collections, for information on how to use the CLI and REST API to create, maintain, and delete scopes and collections.

      For information on using the Backup Service to inspect scopes and collections within backed-up data, and to restore data to specific scopes and collections, see Manage Backup and Restore.

      For information on using scopes and collections with Cross Data Center Replication (XDCR), see Replicate Using Scopes and Collections.

      For a complete reference to the CLI command for managing scopes and collections, see collection-manage.

      For reference pages for the REST API commands that manage scopes and collections, see Scopes and Collections REST API.

      For the cbstats reference pages for scopes and collections, see scopes, scopes-details, collections, and collections-details.

      As indicated above, it is possible to create, maintain, and delete scopes and collections using {sql++}. See the SQL++ Language Reference for further details.

      Working with Scopes and Collections from Couchbase SDKs

      The 3.x API versions of Couchbase SDKs work with scopes and collections. For information, see the Java Howto doc.