Release Notes for Couchbase Server 5.5.x

MB-35717

Fixed handling of SSL_ERROR_WANT_WRITE.

MB-35702

SSL write failures may not reset the connection in the write phase. This has been fixed by propagating SSL write errors from sendmsg.

MB-35593

Extended logging for SSL read/write failures.

MB-35473

Fixed an issue that caused a Write Bucket Failure and corrupted the vBucket file.

MB-35748

Fixed an issue where, under unknown circumstances, leader_lease_acquirer would hang trying to shutdown worker.

MB-29525

Added the ability to filter internal users for logging audit events.

MB-36600

Fixed an issue in the index service which caused it to crash continuously.

MB-36575

Fixed an issue that caused the Plasma storage engine to grow indefinitely.

MB-36298

A StreamEnd message for a vBucket can be processed before a StreamBegin message in feed’s backch. This lead to inconsistent book-keeping and caused the index service to enter an infinite loop. This has been fixed.

MB-36273

The ForestDB reserve bitmap revision number was not set correctly and caused cbbackupmgr merge to fail. This has been fixed.

MB-36052

Index creation failed as the index service was unable to create index replicas. This has been fixed.

MB-35804

Running COUNT on an index FullScan returned incorrect results when it included DESC order. This has been fixed by updating the code to handle Inclusion getting flipped due to the DESC order.

MB-35323

Fixed an issue in ForestDB where renaming a compacted file led to the file getting deleted.

MB-34455

Fixed a memory allocation failure in Forestdb that caused a system crash under pressure.

MB-34167

Fixed an issue where new index builds remained stuck if indexes on MAINT_STREAM were dropped before indexes on INIT_STREAM during rebalance.

MB-34165

Fixed an issue where the index node crashed with too many open files.

MB-36433

Fixed an issue that caused a drop in throughput for array queries.

MB-35811

Fixed a connection pool leak observed when querying system:indexes.

MB-35732

Some proactive fixes around covered queries delivered in release 5.5.5 were observed to cause regressions and hence have been reverted in this release. The reverted issues are MB-35392, MB-34666, and MB-34660.

MB-34750

Avoid IntersectScan when the index is a subset.

MB-34426

Fixed an issue where the query engine returned incorrect results when using a partition index and the ORDER BY clause.

MB-36391

The Mochiweb acceptors would hang forever if TLS handshake froze. This has been fixed by setting a timeout for TLS handshake.

MB-36316

The couchbase-cli utility now works with secure connection and CA verification.

MB-34297

Fixed an issue so that XDCR no longer deletes replication specs incorrectly during transient rebalancing states. If the target bucket is removed, XDCR will still remove the replication spec using a non-topology-change-detector module. This removal mechanism is more fail-safe.

MB-34140

Fixed broken rebuild_replicas script.

MB-34507

Fixed an issue where documents with a TTL, containing system XATTRs, and which are compressed, may have their datatype metadata incorrectly set. This prevents any subsequent modification of the XATTRs on the deleted document, until the document is re-created.

MB-34329

Reset snapshot range to be a valid range.

MB-33906

Fixed an issue where the response to a getReplica command could be significantly delayed.

MB-34624

The index service now uses CountTotal for count aggregate on full index scan (leading key, no group by). This improves query performance as the count is precomputed.

MB-34406

Fixed a memory leak caused by snapshot metadata objects linked list references.

MB-34638

Fixed an issue that caused Couchbase Server to take > 130% CPU while idle on Mac OS.

MB-34419

Couchbase Server versions 5.5.3 and 5.5.4 failed to start on Mac OS platform due to an issue in the startup script. The startup script has been updated in version 5.5.5.

Note that Mac OS packages for versions 5.5.3 and 5.5.4 have been removed from the Downloads page. We recommend that you upgrade to the latest maintenance release on a given series to avail the bug fixes and improvements.

MB-34666

Better identifier handling to determine whether an identifier should be covered or not.

MB-34665

Avoid IntersectScan when the index is a subset.

MB-34660

Fixed an issue where the implicit cover of nested array did not work as expected with arbitrary variables.

MB-34544

The query service now generates a special error for subquery build failures and sets the HTTP error code appropriately.

MB-34224

Updated information on adding roles to a user.

MB-34126

The couchbase-cli tool no longer enforces the 24 character password limit.

MB-33675

Following an issue on a remote cluster, XDCR failed to recover and threw an exception.

MB-32658

XDCR failed to replicate files larger than the network usage limit. This has been fixed so the bandwidth throttler can handle large documents with sizes greater than the usage limit.

MB-32222

Fixed an issue where the XDCR log level setting was not retained when upgrading Couchbase Server from version 4.x to newer version.

MB-32218

Added timeout to replication startup sequence.

MB-32217

Fixed the heartbeat mechanism to detect when an XDCR rest server was stuck.

MB-33853

Fixed a race condition that was exposed when a supervisor that is shutting down normally, is for some reason brutally shutdown leaving an orphan child process. This rogue lease acquirer interfered with the operation of the orchestrator causing, for instance, rebalance to fail.

MB-33852

Memcached returned ENGINE NOT_STORED to client after trying to perform arithmetic operations and caused a race condition.

MB-33773

Fixed an issue that caused a race condition between DCP consumer message processing and stream shutdown.

MB-33351

Fixed an issue where the breakpad did not catch the memcached segfault (exit status 139).

MB-32840

Fixed an issue where the audit daemon asserted when rotating an empty audit log causing a crash.

MB-32696

Fixed an issue so that when an on connection error is encountered, certain operations are retried before returning an error.

MB-32685

Introduced a delay in updating the RBAC database revision number to avoid returning AUTH_STALE during a refresh of the RBAC database.

MB-32669

Fixed an issue where a chain of events, active compression of an extended attribute value followed by eviction and then expiry, triggered an exception.

MB-32660

If an append/prepend request contended with another request at the Server, instead of automatically retrying on the Server (as expected), it returned EEXISTs back to the application.

MB-32656

Added audit trail for the connected bucket.

MB-32637

Fixed an issue where CAS value 0 was returned for a successful Append response.

MB-15009

Defragment document StoredValue in addition to blobs to help cases where a large number of documents with a given key size are deleted, and then new documents are allocated with a different key size. In this case the original size class would not be able to be reused, and hence fragmentation could be high in that class.

MB-33664

It was possible to create an index with fewer replicas than the default num_replicas if you used nodes_list in the WITH clause with fewer nodes than the default number of replicas.

MB-33653

Fixed an issue where frequent processing of old create tokens caused high CPU usage on index nodes.

MB-33639

Fixed an issue to terminate all watcher goroutines upon close of MetadataProvider.

MB-32720

The index service was getting stuck during warmup because of coredumps. This has been fixed.

MB-32716

Fixed an issue that caused the index service to reach its quota with very few indexed items.

MB-32657

Fixed an issue to enable creating secondary indexes on binary documents on meta.id(), meta.cas(), and meta.expiration().

MB-32641

The DCP noop interval has been lowered to 20sec to improve the chances of the projector receiving the noop message early and thus avoiding frequent roll backs.

MB-32640

Improved the DCP rollback handling mechanism. Instead of ignoring the disk snapshots before rolling back, the index service will now exhaust trying DCP stream requests with all the disk snapshots before rolling back to 0.

MB-32636

Plasma memory tuner will now increment indexer memory quota only if the indexer process' RSS is below the memory quota.

MB-32635

In a cluster with duplicate index names across buckets, replica repair failed after a node was added back to the cluster. This has been fixed.

MB-33208, MB-33185

Fixed an issue with connection pool leak that caused queries to fail.

MB-33040

The builtin Administrator user can now be authenticated via a client certificate.

MB-33429

The couchbase-cli setting-alert man page has been updated to include the alert-communication-issue flag.

MB-33204

The cbbackupmgr backup --resume incorrectly required full administrator privileges. This has been fixed so that any user with back and restore permissions can also resume a backup.

MB-32671

The cbimport key generator did not handle escaped hash(#) or percent(%) characters correctly.

MB-32659

Fixed an issue where the Time Interval checkbox on the Auto-Compaction page was disabled upon initial page load.

MB-32419

Fixed an issue where cbrestore did not restore Views definitions taken using cbbackup.

MB-32045

Fixed an issue where encrypted XDCR was incorrectly using port 8091.

MB-31763

The unit of interval for XmemSelfMonitorInterval was incorrectly set to millisecond instead of second, causing an increase in the frequency of Xmem self monitor and the spurious Xmem is stuck error. This has been fixed.

MB-31568

Improve XDCR connection management to prevent XDCR replication getting stuck while creating a checkpoint.

MB-32181

If one or more Ephemeral buckets were present, then nonIO background tasks could be incorrectly scheduled, potentially resulting in DCP connection instability (premature disconnection). This has been fixed.

MB-31837

In some cases, the memory allocation failures were not handled causing "Write Commit Failure" errors or corruption in vBucket files.

MB-31835

When data corruption is detected on a data service node, relevant information that can help troubleshoot is now being logged.

MB-31585

Fixed an issue where appending to a document reset the expiration value to 0, disabling TTL expiry.

MB-31454

When IPv6 was enabled, Couchbase Server incorrectly used ports that were outside of the known port range. When firewall rules are in place, requiring the ports to be explicitly whitelisted, this blocked nodes from being added to a cluster.

MB-31570

The data service engine may not send STREAM_END message to consumers if non-infinity end sequence number and cursor dropping occurs. This could cause clients such as cbbackupmgr to hang indefinitely.

MB-31745

An error is seen during a SUM aggregate pushdown when the entry value is greater than MaxInt64.

MB-31696

Fixed an issue so that you can now create secondary indexes on binary documents on meta.id(), meta.cas(), and meta.expiration().

MB-31559

The index service crashed if the data bucket was flushed while the initial index was being built.

MB-31648

On Windows platform, the missing data file (icudtl.dat) caused errors in the Views engine and Eventing service. The data file has been restored.

MB-32195

Fixed a race condition caused by the intersect scan incorrectly interpreting the end of a scan as having already processed all the keys from that scan.

MB-32120

Fixed an issue where the query engine threw an error if the query referenced extended attributes and the result contained non-existent keys.

MB-31991, MB-31990

Following a cluster upgrade to version 5.5, the query service failed to retrieve extended attributes from a bucket when the query was run for the first time due to stale connections. This has been fixed to check and establish new working connections before executing a query following an upgrade.

MB-31615

Fixed an issue where the TRUE predicate was handled incorrectly in the WHERE clause causing incorrect results.

MB-32142

Fixed an issue where the Views UI was unable to process the "undefined" value in a binary JSON document.

MB-32072

All requests originating from the UI were authenticated using token based authentication, even when client certificate authentication is enabled on the Server and a client certificate is configured in the browser. This has been fixed to use client certificates for authentication in such scenarios.

MB-31830

The cbbackupmgr utility would hang and not return an error when trying to backup a cluster in which at least one node was down and had not failed over.

MB-30920

Authenticated users with full administrative privileges were able to access the diagnostic endpoint, /diag/eval remotely, allowing these users to run arbitrary code on the server host. This issue is addressed by only allowing full administrators to access this endpoint from localhost. This mitigates the issue as users that have access to the server host are already in a position to run arbitrary code.

Note that the diag/eval endpoint is authenticated and requires the highest privileges in the system (Full Admin). It is the case that a user with access to this endpoint can already delete, corrupt or coy all the data in the system. Customers can mitigate this issue by tightly controlling access to full administrative privileges.

MB-31141

Fixed an issue where replication from version 4.5.x to version 5.5.x caused corruption on deleted documents leading to inability to rebalance.

MB-30509

In previous releases, the memory consumption of the index service could exceed the index service’s RAM quota setting. This caused problems when the quota was set to an amount that was close to the physical RAM on the Server node, for example this could lead to an out of memory error. This issue has been fixed.

MB-30412

After a rebalance, the index service was stuck waiting for Memcached. This has been fixed by adding a timeout mechanism so the index service does not hang.

MB-30382

Fixed an issue where the index service threw an error if an index was dropped when index mutation was ongoing.

MB-29982

The authentication requests from projector to Memcached now have a fixed timeout to avoid infinite wait time.

MB-30227

Fixed an issue where in-place (offline) upgrades to Couchbase Server on Windows from 5.0.x or 5.1.x to later releases failed.

MB-31269

A race condition was observed when deleting a system bucket if the bucket was in use by a system bucket scan. This is a rare condition where the wrong unlock was used when the scan resumed after the delete operation completed.

MB-31240

When establishing connections, the cbq-engine would hang if the data node did not respond when trying to establish a handshake packet exchange with the data node. This has been fixed and the cbq-engine now throws an error if the connection cannot be established.

MB-31017

An empty array in the index scan caused incorrect results. This was observed when the scan_cap parameter was set to a small number (say 2) and thus, backfill was used.

MB-30946

Fixed an issue where an empty array from index scan did not work as expected when backfill was used.

MB-30868

Fixed an issue where the logs collected from the UI did not use the same salt across all nodes even though they were collected at the same time.

MB-30610

Fixed an issue where the Memcached process was unable to start if IPv6 protocol stack was disabled.

MB-30870

During warmup, one of the indexes with low number of items triggered an issue and caused the internal garbage collector to run forever. Thus causing the entire index service to stall during warmup. This issue was observed when using Plasma storage engine.

MB-30823

Fixed an issue where the index service crashed with a "slice bounds out of range" error.

MB-30672

Fixed an issue where upgrading from 4.x to 5.x version of Couchbase Server caused an outage for GSI standard indexes. This was observed during rolling upgrade from ForestDB to Plasma. When the last ForestDB index service node was taken out of the cluster, the cluster compatibility mode changed to Plasma and the index service restarted.

MB-30413

Uninstalling Couchbase Server on a Debian-based systemd distribution such as Ubuntu 16 using "apt-get remove" and then running "systemctl status couchbase-server" reported that the service still exists but is "masked". This means the service was explicitly disabled in such a way that it could not be started, automatically or by hand. As a result, subsequent installations of any version of Couchbase Server failed as the service was rendered unstartable.

MB-30636

Fixed an issue where an unexpected token error was seen when accessing the Web Console in Couchbase Server version 5.5.0.

MB-30589

Fixed an issue where the warning message on the Web Console showed an incorrect flag for the removal of dedicated bucket port as --remove-port instead of --remove-bucket-port.

MB-23074

Summary: Performance issues may be observed when running Couchbase Server on CentOS 7.3 with kernel 3.10.0-514.6.

MB-17571

Summary: On an undersized node, the default memory quota assigned to all the selected services by the server might result in a failure.

Workaround: Adjust the memory allocations appropriately to work around this issue.

MB-30610

MB-30074

Summary: The data node is in a pending state after memcached crashes.

MB-29809

Summary: The last_modified can be incorrect if a bucket was upgraded and contains old documents.

MB-29227

Summary: Couchbase Server can get into a livelock state due to high checkpoint memory usage.

MB-37643

Summary: When bucket-level expiry is enabled and the bucket(max) TTL is set to more than 30 days, then it uses an expiry offset based from when the Data Service is started, instead of an offset based from the current time. This can result in documents that use the bucket TTL being expired prematurely.

MB-31639

Summary: The cbbackupmgr utility fails to backup a cluster with Eventing service when the user role is set to 'Data Backup & Restore' role.

MB-29841

Summary: The debugger is unable to apply the source map generated by the transpiler. A recent update in the Chrome browser version 66.0.3359.181 is causing this issue. Since Google doesn’t allow previous versions of Chrome to be downloaded, you can download previous versions of Chromium. The debugger works fine with source maps on Chromium version 63.0.3239.0.

Note: Our fix for Chromium is tagged to 69.0.3466.0~157. As per the Chrome release calendar, 69.x is set to release on September 4th 2018. The fix has been merged to Chrome Canary and is available at https://www.google.com/chrome/browser/canary.html

MB-29360

Summary: Multiple mutations are observed for a single document update when using Sync Gateway, leading to the OnUpdate handler being triggered multiple times.

MB-29308

Summary: The eventing service may mark redacted data incorrectly.

MB-29271

Summary: When a rebalance is in progress the Eventing service hangs when memcached is killed on data and eventing nodes.

MB-28555

Summary: The Eventing service currently does not provide the ability to specify a port range. Instead, the OS arbitrarily allocates a port for the Chrome Debugger.

MB-28414

Summary: The eventing service misses some mutations with non-default vBuckets (535,1001 etc).

MB-28120

Summary: The eventing service rebalance progress jumps from 24% to Finish.

MB-28010

Summary: The execution_stats.on_update_failure are not counted in the Failures stats displayed in the Web Console.

MB-27814

Summary: When there are multiple functions being deployed, undeployment does not happen until all functions are deployed.

MB-27559

Summary: Benign panics can be seen in the eventing service logs during undeployment.

MB-29967

Summary: In some circumstances, the Search engine uses more memory than its defined quota.

MB-28847

Summary: The memory usage during index build overshoots the RAM quota.

MB-27429

Summary: Scorch indexes were found to contain duplicate pindexes.

MB-25714

Summary: On Windows platform, a very high memory/CPU usage may be observed when search service is indexing.

MB-30207

Summary: An error is seen during a SUM aggregate pushdown when the entry value is greater than MaxInt64.

MB-30011

Summary: The rebalance progress in percentage during GSI swap rebalance doesn’t always increase linearly.

MB-19869

Summary: Rebalance fails when taking out failed over nodes running views, in certain circumstances.

MB-30354

Summary: Upgrading from 4.x to 5.x version of Couchbase Server causes an outage for GSI standard indexes. This is observed during rolling upgrade from ForestDB to Plasma. When the last ForestDB index service node is taken out of the cluster, the cluster compatibility mode changes to Plasma and index service restarts.

MB-30227

Summary: In-place (offline) upgrades Server on Windows from 5.0.x or 5.1.x to later releases will fail.

Workaround: Manually uninstall 5.0.x / 5.1.x from your Windows machines prior to installing 5.5.0 or later Server releases.

MB-29391, MB-29393

Summary: Large numeric values are reported incorrectly. For example, the modulo operation or a scan for min int64 value returns an incorrect result in case of int64.

MB-26421

Summary: The default "administrator" user doesn’t show up in list of local users displayed on the Web Console under the Security -> Users tab.

MB-30589

Summary: The UI displays an incorrect flag in the warning message about the removal of dedicated port for a bucket. It displays --remove-port instead of --remove-bucket-port.

MB-30007

Summary: It is possible for a CONNECT BUCKET statement to fail temporarily with the error message "The vbucket belongs to another server(0x7)" when rebalancing Data nodes.

Workaround: Retry the operation.

MB-29542

Summary: The Analytics stats for failed record count was incorrect.

MB-28544

MB-29227

Couchbase Server could get into a livelock state due to high checkpoint memory usage.

MB-29205

There were a number of issues in Couchbase Server which caused rebalance to fail from time to time. These have been fixed.

MB-29040

Rebalance failed with an "Invalid Format specified for DCP_DELETION" error when data or eventing nodes were swapped in and out of a cluster.

MB-28868

The index service went through a race condition in the storage engine which led to a crash. The crash was most likely to happen if the index service was restarted with existing build indexes and they continued to process mutations after the index service restarted.

MB-28468

The full-text search service repeatedly attempted to setup DCP streams to non-existing vBuckets.

MB-29235

The Eventing service did not retry bucket operation failures that were determined by the appropriate LCB macro to be retriable.

MB-29147

Rebalance of KV/Eventing nodes can hang if the cluster contains Eventing nodes due to stale authentication info with the GoCB SDK.

MB-28968

Rebalance of data or eventing nodes would hang if the cluster contained Eventing nodes.

MB-28779

If the handler code was not idempotent, the destination bucket contained more docs than expected after a rebalance.

MB-28775

On Windows, orphan processes with .rbf extension were found to be running even after uninstalling Couchbase Server.

MB-28667

Under some circumstances, like a function undergoing bootstrap or buckets referenced by the function (source, metadata, destination) get flushed or deleted, the function could not be undeployed.

MB-28550

After restoring from an enterprise backup of eventing functions, eventing service would not process mutations in the destination cluster.

MB-28520

Deployment fails if the size of the handler code is greater than 1MB.

MB-28315

Redeploying a function resulted in the application log being truncated instead of appending to the existing log.

MB-27679

The Eventing service can crash when processing documents in source buckets with size is greater than 1 MB.

MB-27491

Failed rebalance, when retried, could hang.

MB-27454

Rebalance-in of a data(KV) node after recovery from failover may hang if eventing service is processing mutations.

MB-29405

Rebalance would hang during index service rebalance.

MB-28139

If the index files were corrupted, the index service would continue to crash and required a failover or a manual deletion of the corrupted index files to recover.

MB-27815

The array indexing plan incorrectly included filter covers.

MB-27360

Fixed an issue on Windows platform where a query with more than 1000 characters could be pasted into the cbq shell only the up to 3 times. Subsequent paste operations resulted in a partial query being pasted.

MB-25901

Fixed an issue where the ORDER BY operator proceeded to sort even after being stopped and could lead to crashes.

MB-29980

The logic to confirm unsafe stop rebalance operation on the Web Console has been resurrected.

MB-28337

Cbbackup, cbtransfer and cbrecovery can incorrectly timeout and in rare cases cause data corruption.

MB-27719

The cbrecovery command threw an exception when recovering data from a cluster that had failed over nodes.

MB-27280