WL#4252: NDB transporter send buffer unification
Status: Complete
Existing NDB kernel code uses one (2Mb) send buffer for every configured node. The memory is allocated at data node start. This is an inefficient use of memory, as most will be unused, yet it is necessary to configure the size sufficient to cater for max load on any transporter socket. It also makes it expensive to configure (but not run) extra mysqld nodes for later expansion (ie. this worklog may facilitate removing the need to pre-configure mysqld nodes at all). Change send buffer memory to be dynamically allocated from a pool shared among all the transporters. Send buffer size will then be adjusted dynamically.
New configuration parameters with this worklog: 1. TotalSendBufferMemory (NDBD, API, and MGM sections). This is the total amount of memory to allocate for use as shared send buffer memory among all configured transporters. If not set, it defaults to be the sum of the maximum send buffer size of all configured transporters, plus an extra 32k (one page) per transporter. This maximum is the value of SendBufferMemory for TCP transporters, for SCI it is SendLimit + 16 k, and for SHM it is 20k. This is done as a backward compatibility measure, as this way old configurations will work more or less unchanged, allocating the same amount of memory and with the same amount of send buffer available to each transporter (but without the benefit of being able to have memory that is unused in one transporter available for another one). 2. ReservedSendBufferMemory (NDBD section). This optional parameter, if set, gives an amount of memory (in bytes) that is reserved for connections between data nodes. Ie. this memory will never be allocated to send buffers towards management server or API nodes. This provides a way to protect the cluster against badly behaving API nodes hogging all the send memory causing failure in communications internally in the kernel. 3. OverloadLimit (TCP section). This parameter denotes the amount of unsent data that must be present in the send buffer before the connection is considered overloaded. When overload occurs, transactions that affect the overloaded connection start failing with the error ZTRANSPORTER_OVERLOADED_ERROR (1218) until the overload status passes. Additionally, the TCP configuration parameter "SendBufferMemory" changes its meaning. Before, it was the amount of memory allocated at startup for each configured TCP connection. With this worklog implemented, memory is not allocated dedicated to each transporter. Instead, the value denotes the hard limit for how much memory (out of the total available memory, TotalSendBufferMemory) that may be used by a single transporter. Thus if the sum of SendBufferMemory for all configured transporters may be greater than TotalSendBufferMemory. This is a way to save memory when lots of nodes are configured, as long as not all transporters ever need to use the maximum amount of memory at the same time.
Copyright (c) 2000, 2024, Oracle Corporation and/or its affiliates. All rights reserved.