Developing on Couchbase CE with Node.js

Here you can learn how to install and use Couchbase Node SDK. This tutorial was made using Couchbase Server CE, so you can use this distribution to follow this guide free of charge.

Node SDK Setup

Now that you have Couchbase Server up and running, you will need to allow your applications to interact with it. The Couchbase SDK provides an API for that purpose. Follow the steps indicated below to install it in your projects.

Tested for nvm 0.34.0, npm 6.13.4, Node 12.14.1.

If you haven’t installed Node yet, we recommend you to do so using Node Version Manager (nvm). To install it, you need to download the source code from Github as follows.

curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.34.0/install.sh | bash

After the installation finishes, you need to reopen your current shell session to use the nvm command from the console. You can verify that everything went fine with this command.

node --version

Now, to install Node along with its standard package manager npm type the following in your console.

nvm install --lts

This line will install the Long Term Support (lts) version of Node. To verify the current version installed, use this command.

nvm ls

Some Node packages or add-ons may require to be compiled before installation, so you should install the development tools as follows.

sudo apt install build-essential

With these steps, Node should be ready to use. You can start new projects as shown below.

mkdir new_project
cd new_project
npm init

You will be asked some project metadata, fill in as you need. Once it’s done, the last step would be installing Couchbase SDK.

npm install couchbase

Your Node project is now ready to interact with any Couchbase Server.

For a more extensive install guide, you can follow the Couchbase documentation for Node SDK

Example Project

Most applications we work on, usually require some level of abstraction or framework to interact with data, especially for CRUD operations. In this example project, we will implement a collection of functions capable of manage ratings, users and movies. Something you would need if working on a movie rating site, as IMDb for example.

Populate Your Server

Buckets

Couchbase Server uses Buckets and Collections to store and group documents respectively.

There are three types of buckets according to persistence, and three ways to create them (CLI, REST API, and Web UI). You can have one or more buckets and for each one, you can control access (who can create, delete and list objects in the bucket) and check the logs for access to the bucket and its objects.

Inside each bucket, you can create and use up to 1000 collections (groups) of documents and up to 100 scopes to group those collections. Every bucket has a default collection and scope that will be used if you don’t specifically target a different one.

In this tutorial, we will use a Couchbase Bucket, which is stored both on memory and disk, and three collections, one for each type of document (movies, users, and ratings), grouped under the default scope. Collections and scopes are under Developer Preview so you can only use them through the SDK. Buckets, on the other hand, can be created with the Web UI on port 8091.

Go to the Buckets tab and click on Add Bucket. Now create a bucket named rate-these-movies with 100 Mb using default configurations.

We have already created a sample bucket with the project’s data and saved it to disk using the cbtransfer tool. To import the sample bucket into your server, you must now use the cbrestore tool. These assets are useful to create and restore backups of your buckets or your entire server at any moment.

To import the data open you console and type these comands, make sure to substitute your parameters correctly.

cd /opt/couchbase
cbrestore /path/to/cbb couchbase://hostname:8091 -u username -p password

If everything went well, you should see this message on the console.

[####################] 100.0% (138/estimated 138 msgs)
bucket: b'rate-these-movies', msgs transferred...
       :                total |       last |    per sec
 byte  :               198972 |     198972 |  3161194.6
done

Indexes

Indexes enhance the performance of query and search operations, especially as the buckets grow in size. Creating secondary indexes (GSI) on the document values will also let you perform JOIN operations ON those.

You can manage Indexes through the SDK, this time however we will do it from the Web UI. Click the Query tab, and execute the follow line to create the primary index of our bucket.

CREATE PRIMARY INDEX `movies_primary` ON `rate-these-movies`

Then, execute

CREATE INDEX `movies_secondary_movie` ON `rate-these-movies`(`id_movie`)

CREATE INDEX `movies_secondary_user` ON `rate-these-movies`(`id_user`)

to create the secondary indexes needed to perform JOINs between movies, users and ratings.

You just executed N1QL queries on your server, keep reading to learn how to run them using the SDK as well.

Visualize

The simplest way to verify your data state, and quickly access a particular document, is through Web UI. Go to Buckets tab, and click the Documents button on any bucket.

Click each document for a more extended view, or set some filters to make a specific search.

Using the SDK

Couchbase SDK provides you with multiple ways to manipulate data:

Core operations or key-value operations, are quite basic and will allow you to work with your data similar to how you would do with a dictionary. But, if you want to perform more complex operations like filters or joins, you would have to implement those behaviors on the client-side. Also, they work with the full document, rather than the exact values you may need.
Sub-document operations can target specific values in a document. Use these operations to save bandwidth, and be more efficient when consulting partial data.
N1QL is an expressive, powerful, and complete SQL dialect for querying, transforming, and manipulating JSON data. These queries will be interpreted by the server and transformed into core operations. Most queries will require the creation of indexes to join other buckets or decreasing query latency.

Connect

Let’s see now how to establish a connection to the server using the SDK to open our previously created bucket rate-these-movies.

This step requires credentials, as a shortcut, you could use the ones used to set up the cluster. Although we don’t recommend this for a production deployment, it fits this tutorial purpose. If you wish to create new credentials with specific permissions, you can follow this link.

To gain access to the server, you can use the Cluster class. An instance of this class can be used to open buckets and manage data through queries and other operations.

var cluster = new couchbase.Cluster('couchbase://' + hostname,
                {
                    username: username,
                    password: password
                })
var bucket = cluster.bucket(bucket_name)

You don’t need to explicitly disconnect from the server, this will be performed automatically when the instance falls off your code’s scope.

Core Operations

When you need to simply insert, delete or retrieve a particular document, of which you know its ID, the recommended approach would be to use core operations.

For inserting a document, you can use any of the operations below, the only difference between them is how they react to previously existing documents:

insert will only create the document if the given ID is not found within the database.
replace will only replace the document if the given ID already exists within the database.
upsert will always replace the document, ignoring whether the ID has already existed or not.

Most times, upsert would be the safest choice, let’s use it to add a rating, a standard operation in any rating site.

var rating_json = { movie_id: movie_id, user_id: user_id, value: value }

var answer = await bucket
                    .collection('ratings')
                    .upsert(rating_id, rating_json)
                    .catch((reason) => console.log(reason));
if (answer) {
    console.log('OK')
}

Notice the use of collection to target a specific group of documents. Through this tutorial we will use console.log to print answers and give feedback, if you are working on a web site, that’s where you would render a view with the data returned by Couchbase Server.

Operations like replace or upsert can be used to update an existing document. However, remember this will send the full document to the cluster, so as a rule of thumb, do this only when more than half of the values have changed. Later on, we will explain how to update data more efficiently when changes are minimal.

To retrieve documents previously inserted in a bucket, use the get operation. You can use it now to check the test rating we just inserted in the server.

var answer = await bucket
                    .collection('ratings')
                    .get(rating_id)
                    .catch((reason) => console.log(reason));
if (answer) {
    console.log(answer.value)
}

If a user wishes to remove its rating from our server, use the remove operation. Try it by removing the rating you have been using until now.

var answer = await bucket
                    .collection('ratings')
                    .remove(rating_id)
                    .catch((reason) => console.log(reason));
if (answer) {
    console.log('OK')
}

Sub-document Operations

Apps will commonly need to change data: ratings for instance, or some miss-typed name. Most of the time this means changing a particular value, not an entire document. For example, a user document may contain a name, a country, and an age, but you only want to update the country the user is currently living. When this situation presents, you should use sub-document operations to target those specific values and reduce network traffic.

Code bellow shows you how to retrieve a particular value from a particular user.

var answer = await bucket
                    .collection('users')
                    .lookupIn(user_id, [couchbase.LookupInSpec.get('country')])
                    .catch((reason) => console.log(reason));
if (answer) {
    answer.results.forEach((result) => {
        console.log(result.value)
    })
}

Notice how we target a particular document with lookupIn, then use get to retrieve the value we want, in this case, the country.

On the other hand, if a user moves to another country and wishes to update its profile, you can do something like this.

var answer = await bucket
                    .collection('users')
                    .mutateIn(user_id, [couchbase.MutateInSpec.upsert('country', country)])
                    .catch((reason) => console.log(reason));
if (answer) {
    console.log('OK')
}

Now we use mutateIn to target the document we want to change, and then upsert to modify its country value.

N1QL Queries

These queries allow us to find and work better with associated documents, as usually required by most applications. For example, if we intend to remove a movie, which has ratings referring to it.

Parameters for the query can be passed in an object, grouped in an array. They can get referenced in the query with $ and the position of the array plus 1.

var answer_single = await cluster
                            .query('DELETE FROM `rate-these-movies` USE KEYS $1', { parameters: [movie_id] })
                            .catch((reason) => console.log(reason));
var answer_linked = await cluster
                            .query('DELETE FROM `rate-these-movies` WHERE id_movie=$1', { parameters: [movie_id] })
                            .catch((reason) => console.log(reason));
if (answer_single && answer_linked) console.log('OK')

Appreciate the simplicity and resemblance to an SQL query, just refer the bucket as you would with a table.

Another example, most read operations target a subset of data or require some aggregation or augmentation to be performed. So, once again, we will depend on N1QL queries, in this case, to get the top 5 rated movies along with its average rating.

Since we are joining the data of a bucket with itself we use aliases.

var answer = await cluster
                    .query('SELECT a.name AS name, AVG(b.`value`) AS avg FROM `rate-these-movies` AS a JOIN `rate-these-movies` AS b ON META(a).id=b.id_movie GROUP BY a.name ORDER BY avg DESC LIMIT 5')
                    .catch((reason) => console.log(reason));
if (answer) {
    answer.rows.forEach((row) => {
        console.log(row.name + ' -> ' + row.avg.toFixed(2))
    })
}

Next steps

We recommend you to follow our next tutorials, go to the Getting Started with Couchbase Community Edition page to find the full list.

Also, you could review Couchbase Documentation to learn more about all sorts of topics.