Release Notes for Couchbase Server 6.5

MB-44220

Summary: Fixed an issue where the Analytics driver went into a restart loop when a filesystem corruption was encountered, resulting in the Analytics service exiting with an error.

MB-44215

Summary: The validity bit was used to determine corrupted merged components.

MB-44214

Summary: When making the debug log async, the DCP client log level was incorrectly set to ERROR. Fixed this issue to use the global service log level instead, which defaults to DEBUG.

MB-42918

Summary: If a durable write is performed via INSERT (Add), an existing item may be removed from the hash table if it was deleted and not yet persisted. Another client performing a GET could trigger a fetch of the item from disk, and the returned item could be the logical predecessor of the unpersisted DELETE that was removed from the hash table if it had not yet been persisted. This means that the client would not be able to read their own write. This issue has been fixed.

MB-42610

Summary: If a replica vBucket was promoted to active having only received a partial backfill (a data loss scenario), then a subsequent expiration of an item could expire a pending durable write if it had the same CAS. This caused any future lookups or writes to that key to cause memcached to crash. This issue has been fixed.

MB-42607

Summary: During TLS handshake with the Data Service, if the node certificate required more than 8192 bytes to transmit, then the handshake failed. This has been fixed.

MB-41868

Summary: Fixed an issue where histograms implemented with HDRHistogram stopped reporting when the total samples exceeded 2^31.

MB-41867

Summary: Fixed an infinite loop due to HdrHistogram being reset.

MB-41255

Summary: Fixed an issue where the replication of deletes with XATTRS failed if the affected document was non-resident.

MB-41407

Summary: Fixed a potential crash during eviction for Ephemeral bucket with nruEviction configured. The crash was a result of a race condition leading to heap use after free. This is a rare condition and has only been seen during data race analysis.

MB-41300

Summary: Fixed a potential issue where sanity checks may trigger at Replica and cause a crash when the node receives temporary mutations for Sync Replication.

MB-40543

Summary: Added the ability to re-enable FTS logging in memcached for debug purposes.

MB-40531

Summary: Fixed a performance regression where the HiFi-MRU Item Pager (added in 5.0) would evict more active items compared to previous LRU ItemPager under certain workloads.

MB-38283

Summary: Fixed an issue where Memcached log wrapped due to excessive DCP "no memory" messages.

MB-43921

Summary: Fixed an issue where the listReplicaCount operation timed out causing a failure during index creation.

MB-43517

Summary: The index service now sets a more contextual user-agent in HTTP requests to the cluster manager(ns_server).

MB-43232

Summary: Starting with version 6.5.0, VbSeqnosReader has been updated to process two types of requests: VbSeqnosRequest and VbMinSeqnosRequest. When processing VbSeqnosRequest, if there are any VbMinSeqnosRequest’s, then the VbMinSeqnosRequest’s will be queued back into the requestCh of VbSeqnosReader. However, if the VbSeqnosReader closed by this time, then enqueue would fail and the caller would be waiting for a response indefinitely. This has been fixed to respond to outstanding requests upon exit of VbSeqnosReader.

MB-43132

Summary: During index definition operations, the cluster info cache is updated multiple times. In a cluster with large number of buckets, refreshing the cluster info cache took a long time and slowed down these operations. This has been fixed.

MB-43072

Summary: Fixed an issue where a delay in the connectBucket function seemed to cause the indexer to projector connection time out.

MB-43047

Summary: Increased the size of the change notification channel in leader to avoid possible deadlock in removeListener in gometa leader.

MB-43046

Summary: The projector went into a stream termination loop when trying to stream a near 20 MB document due to redundant doc size checks in projector. This has been fixed.

MB-43041

Summary: Fixed a runtime error caused by invalid memory address or nil pointer derefernce by adding compression correctness checks.

MB-43039

Summary: Fixed a memory growth issue observed when processing many metadata operations.

MB-43384

Summary: Fixed an issue where setupSSL() failed and impacted query execution.

MB-43301

Summary: Fixed an issue where bulk deletes with USE KEYS did not process all keys.

MB-42555

Summary: Fixed an issue causing inconsistent response from a query using USE keys.

MB-42474

Summary: Fixed an issue where index aggregates were being pushed down incorrectly.

MB-41308

Summary: Fixed an issue where parsed.Index() cached the wrong element.

MB-39782

Summary: When passing permission string to cbauth, cbq engine did not remove backticks. This has been fixed.

MB-41634

Summary: Set the index for the serverMap incrementally.

MB-41585

Summary: Fixed an issue of too many open connections by closing old gRPC ClientConns on refresh.

MB-40515

Summary: Fixed an issue where flex index query failed periodically by initializing the index client only when successful.

MB-40125

Summary: Fixed an issue where some nodes were unavailable following a failover.

MB-37811

Summary: Fixed an issue where the Analytics node failed to rebalance in after an IP reassignment. This issue was observed in a Kubernetes environment.

MB-37458

Summary: On Azure, an Analytics cluster with more than one node failed during cluster configuration as the second node was unable to join the cluster. The issue was caused by resolving the name of each node on the node itself, instead of resolving the names to addresses on the caller side. The best-practices document instructs users to assign the public DNS node name to localhost in the hosts file, causing the callers to erroneously attempt to contact nodes on their own loopback interface. This has been fixed.

MB-37358

Summary: The analytics service no longer makes all assigned quota available to the JVM as heap. The service reserves some amount of this quota for JVM overhead, to help prevent the system from becoming unstable due to low memory.

MB-37420

Summary: Starting with this release, Couchbase Server supports node-to-node encryption with wildcard certs in SAN entries.

MB-38145

Summary: Fixed an issue where N1QL generated from Eventing used only indexes that were present at the time of deployment. This issue was only observed with Couchbase Server versions 6.0.3 (build 2895) and 6.5.0 (build 4960).

MB-32437, MB-38033

Summary: In a multi-bucket scenario approaching the new 30 bucket limit, the Eventing service supports only one function per bucket. An error will be thrown when deploying the second function on a given bucket.

MB-38390

Summary: It was impossible to directly upgrade Couchbase Server from versions prior to 5.5.0 to version 6.5.0 if any cluster contained deleted documents. This was caused by DCP Deletion V2 packets being sent regardless of whether the DELETE_TIMES feature was negotiated.

This has been fixed and you can directly upgrade Couchbase Server from versions prior to 5.5.0 to version 6.5.1. See Supported Upgrade Paths for details.

MB-25982

Summary: Fixed an issue where compaction of large vBuckets caused a significant increase in memory usage.

MB-37231

Summary: New file append mode in cbq to append redirected output to the end of an existing file, rather than overwriting the existing file. See cbq: The Command Line Shell for N1QL for details.

MB-38303

Summary: Fixed an issue where the Scorch file merger leaked index snapshot refCounts on index closure.

MB-38142

Summary: An exception in the third-party dependency "RoaringBitmap/roaring" caused a runtime exception (index out of range) in the Search service. This has been fixed by upgrading the roaring SHA to release v0.4.19 which contains the fix.

MB-37498

Summary: Fixed an issue where the cluster name information was lost in settings when adding an index node to the cluster.

MB-36461

Summary: In cases where the input to IN subclause with EVERY quantifier is MISSING or NULL, Analytics and Query engines differ in behavior. The Analytics service treats MISSING or NULL input values(in this case) as equivalent to an empty array, which results in the whole EVERY … IN … expression returning TRUE, while the Query service returns MISSING if the input is MISSING (or NULL if the input is NULL).

Workaround: Use the IS KNOWN predicate to test whether the IN value is not NULL/MISSING.

WHERE (x IS KNOWN) AND (EVERY y IN x SATISFIES …​ END)

MB-37420

Summary: Node-to-node encryption with wildcard certs in SAN entries are not supported in this release.

MB-32528

Summary: When a synchronous durability request is made unrelated operations pipelined on the same socket may be serviced with higher latency, due to MB-10291.

MB-32497

Summary: Memcached connections may hang due to Connection::signalIfIdle triggering a deadlock when used from a different thread.

MB-25982

Summary: Compaction of large vbuckets causes a significant increase in memory usage.

MB-38145

Summary: N1QL generated from Eventing uses only indexes that were present at the time of deployment. Ensure all indexes necessary for queries utilized by the function have been created prior to deploying the function. This issue is only observed with Couchbase Server versions 6.0.3 (build 2895) and 6.5.0 (build 4960).

MB-37042

Summary: The CLI and REST tools for exporting and importing Eventing functions do not support cross compatible syntax when using only the command line, however the UI can import either syntax.

MB-32437

Summary: In a multi-bucket scenario approaching the new 30 bucket limit, the Eventing service supports only one function per bucket. An error will be thrown when deploying the second function on a given bucket.

MB-31639

Summary: The cbbackupmgr utility fails to backup a cluster with Eventing service for a user with "Data Backup & Restore" role.

MB-31039

Summary: A disk usage spike is observed during the initial index build. The disk usage comes down once the log cleaner catches up.

MB-38390

Summary: It is impossible to directly upgrade Couchbase Server from versions prior to 5.5.0 to version 6.5.0 if any cluster contains deleted documents. This is because DCP Deletion V2 packets are sent regardless of whether the DELETE_TIMES feature is negotiated.

Workaround: Upgrade to an intermediate version, such as Couchbase Server 6.0.4+, before upgrading to 6.5.0.

MB-37269

Summary: Upgrading from version 6.0.4 to version 6.5.0 using offline upgrade or rolling online upgrade using graceful failover and delta node does not work. Version 6.0.4 included a per-node configuration migration and it was released after 6.5.0 which doesn’t recognize the migration.

Workaround: To upgrade successfully from version 6.0.4 to 6.5.0, you must use swap rebalance. This will be fixed in the next maintenance release on the 6.5.x series.

MB-33522

Summary: The cbupgrade script fails to upgrade single node IPv6 clusters.

MB-38046

Summary: When performing an online upgrade from Couchbase Server 6.0.3 to Couchbase Server 6.5, some active Couchbase SDK 2.x applications may give an exception 4040: No such prepared statement.

Workaround: Refer to Prepared Statement Upgrades.

MB-37768

Summary: On macOS (and OS-X) the cbc binary needs @rpath to be manually set on the cbc binary, this will allow proper execution of cbc (and related symlinked command line tools).

MB-36750

Summary: The following values for the format parameter of the Analytics Service API are being deprecated in this release and will be removed in a future release: application/x-adm, application/json, application/json;lossless, text/csv.

MB-36702

Summary: Fixed an issue where an analytics query threw an internal error when using to_string() to try to force optimizer to use an index.

MB-36455

Summary: Fixed an issue where the analytics query returned different results (extra 'null' value) when indexnl hint was applied to left outer join.

MB-33764

Summary: In release 6.0.2 and above, only users with "security-admin" role assigned will be allowed to POST an HTTP request to the /controller/uploadClusterCA and /node/controller/reloadCertificate endpoints. In the previous releases, the user had to have "cluster-admin" role assigned in order to POST an HTTP request to /node/controller/reloadCertificate endpoint.

MB-30920

Summary: Couchbase Server exposes '/diag/eval' endpoint which by default is available on TCP/8091 and/or TCP/18091. Authenticated users that have 'Full Admin' role assigned can send arbitrary Erlang code to 'diag/eval' endpoint of the API. The code will be subsequently executed in the underlying operating system with privileges of the user which was used to start Couchbase.

MB-30074

Summary: If the memcached process crashed while indexes were being rebalanced, the memcached process could be failed to be brought online and the rebalance got stuck. This has been fixed and rebalance is now interrupted by memcached failure.

MB-30553

Summary: Fixed an issue where requesting the hash statistic severely affects the response times of replicateTo requests.

MB-32244

Summary: When dealing with non-existent keys, the get operation will now return undefined if the key doesn’t exist and the delete operation will now be treated as a no-op if the key doesn’t exist. Prior to version 6.5 both operations ( get or delete) threw exceptions when applied against non-existent keys. Note, the "language compatibility" setting in the Function Settings dialog allows for backward compatibility to maintaining prior behavior.

MB-33478

Summary: Any kind of collatejson crash caused a downtime, particularly on the mutation path. This has been fixed to recover only from collatejson crashes. After recovering, the docid is logged and the mutation key is skipped, and every such recovered-skip docid is logged as an error on the console.

MB-30207

Summary: Fixed SUM aggregate query for large integers.

MB-33003

Summary: Corrected the port information in the post installation message.

MB-30227

Summary: Fixed an issue where in-place (offline) upgrades to Couchbase Server on Windows from 5.0.x or 5.1.x to later releases failed.

MB-27129

Summary: The install experience on Mac OS has been improved and starting this release, a disk image installer is available for Mac OS.

MB-27429

Summary: Fixed an issue where Scorch indexes were found to contain duplicate pindexes.

MB-35840

Summary: The View index build progress information was missing on the UI and has been fixed.

MB-33207

Summary: Fixed an issue where the CB_CLUSTER environment variable doesn’t handle the prefix couchbase://` or no prefix at all. The fix ensures it behaves the same as using the --cluster argument with the cbbackupmgr tool.

MB-29974

Summary: Fixed an issue where full-text search index aliases were not backed up with the cbbackup tool.

MB-29962

Summary: Fixed an issue where full-text search index aliases were not backed up with the cbbackupmgr tool.