Deployment Considerations

The following guide describes a set of recommended best practices for a production deployment of Couchbase Mobile.

Sync Gateway Security

Authentication

In a Couchbase Mobile production deployment, administrators typically perform operations on the Admin REST API. If Sync Gateway is deployed on an internal network, you can bind the adminInterface of Sync Gateway to the internal network. In this case, the firewall should also be configured to allow external connections to the public interface port.

To access the Admin REST API from an entirely different network or from a remote desktop we recommend to use SSH tunneling.

Authorization

In addition to the Admin REST API, a user can be assigned to a role with additional privileges. The role and the user assigned to it can be created in the configuration file. Then, the Sync Function’s requireRole() method can be used to allow certain operations only if the user has that role.

Data Model Validation

In a NoSQL database, it is the application’s responsibility to ensure that the documents are created in accordance with the data model adopted throughout the system. As an additional check, the Sync Function’s throw() method can be used to reject documents that do not follow the pre-defined data model.

HTTPS

You can run Sync Gateway behind a reverse proxy, such as NGINX, which supports HTTPS connections and route internal traffic to Sync Gateway over HTTP. The advantage of this approach is that NGINX can proxy both HTTP and HTTPS connections to a single Sync Gateway instance.

Alternatively, Sync Gateway can be configured to only allow secure HTTPS connections, if you want to support both HTTP and HTTPS connections you will need to run two separate instances of Sync Gateway.

Couchbase Server Auth

There are two methods to securely connect a Sync Gateway instance to a Couchbase Server cluster. Each method is discussed below in more detail.

Username and Password

The username and password of the RBAC user are specified in the Sync Gateway configuration file. This method is used in the getting started section.

X.509 Certificates

Sync Gateway 2.1 adds the ability to use X.509 certificates to authenticate against Couchbase Server 5.5 or higher. This functionality can be used instead or in addition to the existing authentication method which is to specify a username and password in the configuration file.

X.509 Certificates for Couchbase Server

You must first generate the cluster certificate and node certificates. Instructions to create and install those certificates can be found here (step 1 to 13).

X.509 Certificates for Sync Gateway

The client certificates need to be signed by the Couchbase cluster certificate. So in order to generate certificates for Sync Gateway, we will use the output files from the previous section.

You should now have a folder called SSLCA. From the parent directory (i.e ../SSLCA), follow the steps below.

  1. Create a new file called ssl.conf with the following.

    [ req ]
    default_bits        = 2048
    distinguished_name  = req_distinguished_name
    req_extensions      = req_ext
    prompt              = no
    
    [ req_distinguished_name ]
    countryName         = US
    stateOrProvinceName = California
    localityName        = San Jose
    organizationName    = XYZ Inc
    commonName          = travel-sample
    
    [ req_ext ]
    subjectAltName = @alt_names
    [alt_names]
    DNS.1   = node1-user1.xyz.com
    DNS.2   = node1-user2.xyz.com

    When using X.509 certificates for authentication, Couchbase Server searches for the RBAC username in the certificate sent by the client. In the X.509 Certificates for Couchbase Server instructions, you specified commonName as the field to use for the RBAC username. Hence, the commonName field is set to travel-sample.

  2. Create a new file called client.sh with the following. This script creates a new client certificate.

    export TOPDIR=SSLCA
    export ROOT_DIR=rootdir
    export NODE_DIR=nodedir
    export INT_DIR=intdir
    
    export CLIENT_INT_DIR=client_intdir
    export CLIENT_CERT_DIR=client_certdir
    
    cd ${TOPDIR}
    mkdir ${CLIENT_INT_DIR}
    mkdir ${CLIENT_CERT_DIR}
    
    cd ${CLIENT_INT_DIR}
    openssl genrsa -out int.key 2048
    openssl req -new -key int.key -out int.csr -subj '/C=US/O=XYZ/CN=XYZIntCA'
    cat <<EOF >> ./v3_ca.ext
    subjectKeyIdentifier = hash
    authorityKeyIdentifier = keyid:always,issuer:always
    basicConstraints = CA:true
    EOF
    openssl x509 -req -in int.csr -CA ../${ROOT_DIR}/ca.pem -CAkey ../${ROOT_DIR}/ca.key \
    -CAcreateserial -CAserial ../${ROOT_DIR}/rootCA.srl -extfile ./v3_ca.ext \
    -out int.pem -days 365
    
    cd ../${CLIENT_CERT_DIR}
    openssl genrsa -out client.key 2048
    openssl req -new -key client.key -out client.csr -config ../../ssl.conf
    openssl x509 -req -in client.csr -CA ../${CLIENT_INT_DIR}/int.pem \
    -CAkey ../${CLIENT_INT_DIR}/int.key \
    -CAcreateserial -CAserial ../${CLIENT_INT_DIR}/intermediateCA.srl \
    -out client.pem -days 365 \
    -extfile ../../ssl.conf -extensions req_ext
    cat client.pem ../${CLIENT_INT_DIR}/int.pem > chain.pem
  3. Run the script.

    $ ./client.sh
  4. Create a new file called sync-gateway-config.json with the following.

    {
      "databases": {
        "db": {
          "server": "http://localhost:8091",
          "bucket": "default",
          "users": { "GUEST": { "disabled": false, "admin_channels": ["*"] } },
          "certpath": "./SSLCA/client_certdir/chain.pem",
          "keypath": "./SSLCA/client_certdir/client.key",
          "cacertpath": "./SSLCA/root/ca.pem"
        }
      }
    }

    As shown above, certificate based authentication on Sync Gateway is enabled with the following properties in the configuration file.

Log Rotation

Built-in log rotation

By default, Sync Gateway outputs the logs to standard out with the "HTTP" log key and can also output logs to a file. Prior to 1.4, the two main configuration options were log and logFilePath at the root of the configuration file.

{
    "log": ["*"],
    "logFilePath": "/var/log/sync_gateway/sglogfile.log"
}

In Couchbase Mobile 1.4, Sync Gateway can now be configured to perform log rotation in order to minimize disk space usage.

Log rotation configuration

The log rotation configuration is specified under the logging key. The following example demonstrates where the log rotation properties reside in the configuration file.

{
  "logging": {
    "default": {
      "logFilePath": "/var/log/sync_gateway/sglogfile.log",
      "logKeys": ["*"],
      "logLevel": "debug",
      "rotation": {
        "maxsize": 1,
        "maxage": 30,
        "maxbackups": 2,
        "localtime": true
      }
    }
  },
  "databases": {
    "db": {
      "server": "walrus:data",
      "bucket": "default",
      "users": {"GUEST": {"disabled": false,"admin_channels": ["*"]}}
    }
  }
}

As shown above, the logging property must contain a single named logging appender called default. Note that if the "logging" property is specified, it will override the top level log and logFilePath properties.

The descriptions and default values for each logging property can be found on the Sync Gateway configuration page.

Example Output

If Sync Gateway is running with the configuration shown above, after a total of 3.5 MB of log data, the contents of the /var/log/sync_gateway directory would have 3 files because maxsize is set to 1 MB.

/var/log/sync_gateway
├── sglogfile.log
├── sglogfile-2017-01-25T23-35-23.671.log
└── sglogfile-2017-01-25T22-25-39.662.log

Windows Configuration

On MS Windows logFilePath supports the following path formats.

"C:/var/tmp/sglogfile.log"
`C:\var\tmp\sglogfile.log`
`/var/tmp/sglogfile.log`
"/var/tmp/sglogfile.log"

Log rotation will not work if logFilePath is set to the path below as it is reserved for use by the Sync Gateway Windows service wrapper.

C:\Program Files (x86)\Couchbase\var\lib\couchbase\logs\sync_gateway_error.log

Deprecation notice

The current proposal is to remove the top level log and logFilePath properties in Sync Gateway 2.0. For users that want to migrate to the new logging config to write to a log file but do not need log rotation they should use a default logger similar to the following:

{
    "logging": {
        "default": {
            "logFilePath": "/var/log/sync_gateway/sglogfile.log",
            "logKeys": ["*"],
            "logLevel": "debug"
        }
    }
}

OS log rotation

In production environments it is common to rotate log files to prevent them from taking too much disk space, and to support log file archival.

By default Sync gateway will write log statements to stderr, normally stderr is redirected to a log file by starting Sync Gateway with a command similar to the following:

sync_gateway sync_gateway.json 2>> sg_error.log

On Linux the logrotate tool can be used to monitor log files and rotate them at fixed time intervals or when they reach a certain size. Below is an example of a logrotate configuration that will rotate the Sync Gateway log file once a day or if it reaches 10M in size.

/home/sync_gateway/logs/*.log {
    daily
    rotate 1
    size 10M
    delaycompress
    compress
    notifempty
    missingok

The log rotation is achieved by renaming the log file with an appended timestamp. The idea is that Sync Gateway should recreate the default log file and start writing to it again. The problem is Sync Gateway will follow the renamed file and keep writing to it until Sync gateway is restarted. By adding the copy truncate option to the logrotate configuration, the log file will be rotated by making a copy of the log file, and then truncating the original log file to zero bytes.

/home/sync_gateway/logs/*.log {
    daily
    rotate 1
    size 10M
    copytruncate
    delaycompress
    compress
    notifempty
    missingok
}

Using this approach there is a possibility of loosing log entries between the copy and the truncate, on a busy Sync Gateway instance or when verbose logging is configured the number of lost entries could be large.

In Sync Gateway 1.1.0 a new configuration option has been added that gives Sync Gateway control over the log file rather than relying on stderr. To use this option call Sync Gateway as follows:

sync_gateway -logFilePath=sg_error.log sync_gateway.json

The logFilePath property can also be set in the configuration file at the server level.

If the option is not used then Sync Gateway uses the existing stderr logging behavior. When the option is passed Sync Gateway will attempt to open and write to a log file at the path provided. If a Sync Gateway process is sent the SIGHUP signal it will close the open log file and then reopen it, on Linux the SIGHUP signal can be manually sent using the following command:

pkill -HUP sync_gateway

This command can be added to the logrotate configuration using the 'postrotate' option:

/home/sync_gateway/logs/*.log {
    daily
    rotate 1
    size 10M
    delaycompress
    compress
    notifempty
    missingok
    postrotate
        /usr/bin/pkill -HUP sync_gateway > /dev/null
    endscript
}

After renaming the log file logrotate will send the SIGHUP signal to the sync_gateway process, Sync Gateway will close the existing log file and open a new file at the original path, no log entries will be lost.