Backpressure is a design decision
Every system that accepts work faster than it can complete it must do something. There are only a few options, and the interesting fact is that most systems choose one without deciding.
The four options
- Drop. Reject the work and tell the caller. Cheap, and correct for work that is itself disposable.
- Buffer. Hold the work in a queue. Cheap to implement, and it converts a throughput problem into a latency problem that surfaces later, detached from its cause.
- Block. Stop reading until the work is done. This pushes the decision upstream, which is either exactly right or exactly wrong depending on what is upstream.
- Degrade. Do less work per unit input. Hard to build, and sometimes the only honest option.
The unbounded queue trap
The most common implicit choice is to buffer, with no limit. It looks like it works because nothing is ever rejected, and it hides the overload until the queue is large enough that the system is far past its capacity and the latency has grown beyond anything a caller will wait for. By then the queue is a liability, not a buffer.
A queue with no bound is not a buffer. It is an admission that you have not decided what to do when you are full.
Deciding on purpose
The question to answer is what the system's contract with its callers is. If callers can retry, dropping is fine and fast. If they cannot, blocking upstream is the honest signal. If some of the work is more important than the rest, that is the case for prioritisation rather than a single queue.
Whichever you pick, pick it explicitly, and make the limit a number you chose rather than a number that accumulated.