Eventing Service Memory Quota
- concept
The Eventing Service memory quota does not enforce a hard memory limit on the Eventing subsystem, including worker processes.
This page explains how the memory quota actually works, how the Eventing Service distributes it across workers, and its limitations in non-containerized environments.
About the Eventing Service Memory Quota
The Eventing Service uses the quota for:
-
Worker Queue Sizing: Controls the maximum size of each worker’s input queue.
-
Garbage Collection (GC) Triggering: Determines when the Eventing Service invokes the JavaScript environment’s GC to reclaim memory.
The memory quota is not an absolute memory cap. The quota does not restrict the total memory consumed by JavaScript runtime heaps or Eventing processes.
The Eventing Service divides its total memory quota uniformly across all workers in all deployed functions:
Per-Worker quota = Total memory quota ÷ Total number of workers
For example, if you had a memory quota of 256 MB, with 4 total workers, each worker would use 64 MB of memory, or 256 MB ÷ 4 workers.
Each worker receives an equal share of the total quota, regardless of the actual memory usage of individual functions.
As you deploy more functions or add workers per function, the per-worker quota decreases proportionally. A smaller per-worker quota can affect performance if individual functions require substantial memory for processing.
In VM or bare-metal deployments, Eventing Functions can exceed the configured quota at runtime. Understanding this behavior helps you to appropriately size and monitor your Eventing Functions in production environments.
Worker Queue Sizing
Each worker maintains a bounded input queue that receives changes to documents from the Database Change Protocol (DCP) stream.
The per-worker quota determines the maximum size of this queue.
When the queue exceeds this size, the Eventing Service throttles the DCP stream to prevent unbounded memory growth.
Throttling temporarily pauses the mutation flow, ensuring workers do not become overwhelmed when processing lags behind the mutation rate.
Garbage Collection (GC) Triggering
The Eventing Service’s garbage collection (GC) frees memory occupied by unreachable objects that are no longer in use by Eventing Functions.
Garbage collection reclaims memory only from objects that are no longer in use. Memory held by live objects in the JavaScript runtime heap remains allocated, so total usage may exceed the per-worker quota.
When memory usage approaches the per-worker quota limit, the JavaScript runtime may trigger garbage collection to reclaim memory.
The Eventing Service invokes a stop-the-world JavaScript runtime garbage collection to reduce memory inside the JavaScript runtime isolate running each Eventing Function.
The garbage collector may delay the stop-the-world collection because it’s optimized for throughput.
The runtime might not reclaim memory without a delay, which can temporarily affect the Eventing Service’s memory consumption.
Limitations and Considerations
To prevent operational issues while using the Eventing Service, keep the following in mind while configuring the memory quota:
Not a Hard Limit
The Eventing Service memory quota does not restrict total memory usage.
JavaScript runtime heaps may still contain live (non-garbage) objects that:
-
Garbage collection cannot reclaim.
-
The quota does not include.
-
Continue to consume memory beyond the configured limit.
As a result, setting a memory quota does not guarantee that Eventing Functions remain within that limit.
Memory Management in Containerized and Non-Containerized Environments
In a non-containerized environment, such as bare-metal servers, virtual machines, or EC2 instances, the Eventing runtime deploys 2 classes of processes:
-
The eventing-producer
-
Multiple eventing-consumer processes
These processes effectively have no hard upper bound on memory usage. Collectively, these processes can consume all available system memory.
This can lead to unintended consequences, such as out-of-memory (OOM) conditions that might trigger the OS OOM killer. The OS OOM killer can cause abrupt process termination resulting in service downtime. For this reason, proper system sizing is critical and should not be overlooked. This includes planning total system memory as well as the number of workers configured across all Eventing Functions.
In a containerized environment, all the Eventing runtime processes, like the eventing-producer and eventing-consumer, run within containers governed by cgroups with explicit memory limits. In this case, memory enforcement occurs at the cgroup level and not at the overall host system level. As a result, the underlying system constrains the Eventing processes based on the configured container memory limits, providing more predictable and controlled resource usage.
Worker Sizing Impacts Memory
When configuring the memory quota for the Eventing Service, consider the per-worker quota when determining the number of workers. Avoid over-provisioning workers when Eventing Functions require substantial memory.
For best results, try to balance your worker count against your available memory.
Quota Applies Per Worker
The memory quota does not isolate memory usage between Eventing Functions. All workers draw from the same total memory quota.
A memory-intensive function can reduce the memory available to other Eventing Functions.
Eventing does not support per-function memory limits or reservations.
Best Practices
Follow these recommendations to effectively manage Eventing memory usage:
Monitor Memory Usage
-
Use system-level monitoring tools to track actual memory consumption.
-
Monitor memory usage for the Eventing process and individual workers.
-
Configure alerts for memory thresholds well below system limits.
Test in Representative Environments
-
Test Eventing Functions under production-like load conditions.
-
Measure actual memory consumption during peak processing.
-
Validate that memory usage remains within acceptable limits.
Use cgroups for Enforcement
-
Consider containerized deployments or cgroup-based memory limits in production environments.
-
Use container platforms such as Docker and Kubernetes to provide hard memory limits.
-
Use cgroups to enforce OS-level memory restrictions. This approach adds a safety layer beyond the Eventing Service memory quota.