MySQL 8.3.0
Source Code Documentation
event_reader.h
Go to the documentation of this file.
1/* Copyright (c) 2018, 2023, Oracle and/or its affiliates.
2
3 This program is free software; you can redistribute it and/or modify
4 it under the terms of the GNU General Public License, version 2.0,
5 as published by the Free Software Foundation.
6
7 This program is also distributed with certain software (including
8 but not limited to OpenSSL) that is licensed under separate terms,
9 as designated in a particular file or component or in included license
10 documentation. The authors of MySQL hereby grant you an additional
11 permission to link the program and your derivative works with the
12 separately licensed software that they have included with MySQL.
13
14 This program is distributed in the hope that it will be useful,
15 but WITHOUT ANY WARRANTY; without even the implied warranty of
16 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
17 GNU General Public License, version 2.0, for more details.
18
19 You should have received a copy of the GNU General Public License
20 along with this program; if not, write to the Free Software
21 Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */
22
23/**
24 @file event_reader.h
25
26 @brief Contains the class responsible for deserializing fields of an event
27 previously stored in a buffer.
28*/
29
30#ifndef MYSQL_BINLOG_EVENT_EVENT_READER_H
31#define MYSQL_BINLOG_EVENT_EVENT_READER_H
32
33#include <list>
34#include <map>
35#include <string>
36#include <vector>
39
40/// @addtogroup GroupLibsMysqlBinlogEvent
41/// @{
42
43namespace mysql::binlog::event {
44
45#define PRINT_READER_STATUS(message) \
46 BAPI_PRINT("debug", (message ": m_buffer= %p, " \
47 "m_limit= %llu, " \
48 "m_length= %llu, " \
49 "position()= %llu", \
50 m_buffer, m_limit, m_length, Event_reader::position()))
51
52/**
53 Event_reader class purpose is to avoid out-of-buffer reads when deserializing
54 binary log events and increase robustness when dealing with corrupted event
55 buffers.
56
57 The Event_reader is composed by a pointer to the beginning of the serialized
58 event buffer (m_buffer), a variable containing the buffer length (m_length), a
59 cursor pointer that tells the current position to be read from the buffer
60 (m_ptr) and the buffer limit the reader shall respect (m_limit <= m_length).
61
62 All buffer reading functions shall move the cursor forward.
63
64 Before reading from the buffer, the Event_reader will check if the amount of
65 bytes expected to be read are less or equal to the remaining bytes to read:
66
67 remaining = m_limit - (m_ptr - m_buffer)
68
69 When there are no enough bytes to read from the buffer, Event_reader enters
70 in error state, so its owner can take an action.
71*/
72
74 public:
75 /**
76 Event_reader constructor.
77
78 It sets the cursor to the first position of the buffer.
79
80 @param[in] buffer buffer holding a serialized event
81 @param[in] length known buffer length.
82 */
83 Event_reader(const char *buffer, unsigned long long length)
89
90 /**
91 Returns if the Event_reader is in an error state or not.
92
93 @retval true if the Event_reader is in error state.
94 @retval false if the Event_reader is not in error state.
95 */
96 bool has_error() {
97 BAPI_PRINT("debug", ("m_error= %s", m_error ? m_error : "nullptr"));
98 return m_error != nullptr;
99 }
100
101 /**
102 Returns the pointer to the error message.
103
104 @return the pointer to the error message when Event_reader is in error
105 state, or a nullptr otherwise.
106 */
107 const char *get_error() { return m_error; }
108
109 /**
110 Sets Event_reader error state by setting the error message.
111
112 @param[in] error pointer to the error message.
113 */
114 void set_error(const char *error);
115
116 /**
117 Returns the Event_reader buffer length.
118
119 Note: the buffer length might be larger than reader allowed buffer limit,
120 but the Event_reader will enter error state when trying to read above the
121 limit.
122
123 Example: an event buffer may contain the serialized event + checksum. The
124 event reader object will be configured with a buffer length that contains
125 both the serialized event and the checksum information, but once
126 Log_event_footer is instantiated, it shall adjust the event reader buffer
127 limit to the buffer position right before the checksum. This will avoid some
128 event deserialization relying on event buffer size to assume the checksum as
129 serialized event content.
130
131 @return the Event_reader buffer length.
132 */
133 unsigned long long length() { return (m_length); }
134
135 /**
136 Sets Event_reader buffer length and limit.
137
138 The length of the buffer should only be set to values greater or equal to
139 the current buffer length. Trying to set the length to less than current
140 buffer length will make the Event_buffer to enter error state.
141
142 The length is initially set in Event_reader constructor to
143 LOG_EVENT_MINIMAL_HEADER_LEN by the Log_event_header when instantiating it.
144 This should be enough to read the event header and determine the correct
145 buffer length. The Log_event_header will adjust the Event_reader length by
146 calling this function based on the value of event data_written header field.
147
148 @param[in] length the new Event_reader buffer length.
149 */
150 void set_length(unsigned long long length);
151
152 /**
153 Shrinks the Event_reader buffer limit.
154
155 This function is used by Log_event_footer to remove the checksum payload (if
156 necessary) from the serialized event size, as many event types rely on the
157 serialized event size to determine the size of some fields.
158
159 @param[in] bytes the amount of bytes to shrink the Event_reader buffer
160 length.
161 */
162 void shrink_limit(unsigned long long bytes);
163
164 /**
165 Returns the Event_reader buffer pointer.
166
167 @return the Event_reader buffer pointer.
168 */
169 const char *buffer() { return m_buffer; }
170
171 /**
172 Returns a pointer to the Event_reader cursor (next position to be read by
173 the Event_reader functions).
174
175 @return the pointer to the Event_reader cursor.
176 */
177 const char *ptr() { return m_ptr; }
178
179 /**
180 Returns a pointer to the Event_reader cursor (next position to be read) and
181 moves the cursor forward.
182
183 This function is used when the buffer contains a field of a known size and
184 the deserialization procedure must keep the pointer to the field but moving
185 the cursor to after it.
186
187 @param[in] length the amount of bytes to move the cursor forward.
188
189 @return the pointer to the Event_reader cursor before forwarding it.
190 */
191 const char *ptr(unsigned long long length);
192
193 /**
194 Returns the current Event_reader cursor position in bytes.
195
196 @retval m_limit if cursor position is invalid.
197 @retval position current Event_reader cursor position (if valid).
198 */
199 unsigned long long position() {
200 return m_ptr >= m_buffer ? m_ptr - m_buffer : m_limit;
201 }
202
203 /**
204 Returns the amount of bytes still available to read from cursor position.
205
206 @return the amount of bytes still available to read.
207 */
208 unsigned long long available_to_read() {
210 return m_limit - position();
211 }
212
213 /**
214 Returns if the Event_reader can read a given amount of bytes from cursor
215 position.
216
217 @param bytes the amount of bytes expected to be read.
218
219 @retval true if the Event_reader can read the specified amount of bytes.
220 @retval false if the Event_reader cannot read the specified amount of bytes.
221 */
222 bool can_read(unsigned long long bytes) {
223 return (available_to_read() >= bytes);
224 }
225
226 /**
227 Moves cursor to a given absolute buffer position and returns the pointer to
228 the cursor.
229
230 @param position the position to jump to.
231
232 @retval pointer a pointer to the new cursor position.
233 @retval nullptr if the position is out of buffer boundaries.
234 */
235 const char *go_to(unsigned long long position);
236
237 /**
238 Moves the buffer position forward to a given relative position and returns
239 the pointer to the buffer on the specified position.
240
241 @param bytes the amount of bytes to move forward.
242
243 @retval pointer a pointer to the new buffer position.
244 @retval nullptr if the cursor is out of buffer boundaries.
245 */
246 const char *forward(unsigned long long bytes) {
247 BAPI_PRINT("debug", ("Event_reader::forward(%llu)", bytes));
248 return go_to((m_ptr - m_buffer) + bytes);
249 }
250
251 /**
252 Copies a basic type - bool, char, int, long, double, etc - from the buffer,
253 moves the cursor forward the number of bytes returned by sizeof(T)) and
254 returns the copied value.
255
256 @retval value the T copied from the cursor position.
257 @retval 0 if the cursor was out of buffer boundaries.
258 */
259 template <class T>
260 T memcpy() {
261 PRINT_READER_STATUS("Event_reader::memcpy");
262 if (!can_read(sizeof(T))) {
263 set_error("Cannot read from out of buffer bounds");
264 BAPI_PRINT("debug", ("Event_reader::memcpy(): "
265 "sizeof()= %zu",
266 sizeof(T)));
267 return 0;
268 }
269 T value = 0;
270 ::memcpy((char *)&value, m_ptr, sizeof(T));
271 m_ptr = m_ptr + sizeof(T);
272 return value;
273 }
274
275 /**
276 Copies a basic arithmetic type - uint8_t, [u]int16_t, [u]int32_t,
277 [u]int64_t - from the buffer, moves the cursor forward using specified bytes
278 parameter (or the number of bytes returned by sizeof(T) when not specified)
279 and returns the copied value transformed from little endian if necessary).
280
281 @param[in] bytes the amount of bytes to read from the buffer (and to move
282 forward). When not specified, will use sizeof(T).
283
284 @retval value the T copied from the cursor position.
285 @retval 0 if the cursor was out of buffer boundaries or there was no memory
286 to allocate to the new string..
287 */
288 template <typename T>
289 T read(unsigned char bytes = sizeof(T)) {
290 PRINT_READER_STATUS("Event_reader::read");
291 if (!can_read(bytes)) {
292 set_error("Cannot read from out of buffer bounds");
293 BAPI_PRINT("debug", ("Event_reader::read(): "
294 "sizeof()= %zu, bytes= %u",
295 sizeof(T), bytes));
296 return 0;
297 }
298 T value = 0;
299 ::memcpy((char *)&value, m_ptr, bytes);
300 m_ptr = m_ptr + bytes;
301 return (bytes > 1) ? letoh(value) : value;
302 }
303
304 /**
305 Returns a pointer to a new string which is a duplicate of the input string.
306 The terminating null character is added. See: bapi_strndup().
307
308 @param[in] length the amount of bytes to read from the buffer (and to move
309 forward).
310
311 @retval pointer the T pointer from the cursor position.
312 @retval nullptr if the cursor was out of buffer boundaries.
313 */
314 template <typename T>
315 T strndup(size_t length) {
316 PRINT_READER_STATUS("Event_reader::strndup");
317 if (!can_read(length)) {
318 BAPI_PRINT("debug", ("Event_reader::strndup(%zu)", length));
319 set_error("Cannot read from out of buffer bounds");
320 return nullptr;
321 }
322 T str;
323 str = reinterpret_cast<T>(bapi_strndup(m_ptr, length));
324 m_ptr = m_ptr + length;
325 return str;
326 }
327
328 /**
329 Copies from the cursor to an already existent (and allocated) buffer and
330 moves forward the cursor.
331
332 @param[out] destination a pointer to the destination buffer.
333 @param[in] length the amount of bytes to read from the buffer (and to move
334 forward).
335 */
336 template <typename T>
337 void memcpy(T destination, size_t length) {
338 PRINT_READER_STATUS("Event_reader::memcpy");
339 if (!can_read(length)) {
340 BAPI_PRINT("debug", ("Event_reader::memcpy(%zu)", length));
341 set_error("Cannot read from out of buffer bounds");
342 return;
343 }
344 ::memcpy(destination, m_ptr, length);
345 m_ptr = m_ptr + length;
346 }
347
348 /**
349 Allocates memory to a destination buffer, copies from the cursor to the
350 destination buffer using memcpy() and moves forward the cursor.
351
352 This function is useful for pairs of fields when a first field describes the
353 second field size and the deserialization procedure must allocate a buffer
354 for the second field and then copy the event buffer content to the new
355 allocated buffer.
356
357 Before implementing this function and the Event_reader, the deserialization
358 process did like:
359
360 memcpy(length, ptr, sizeof(length);
361 ptr+=sizeof(length);
362 field = malloc(length);
363 memcpy(field, ptr, length);
364
365 Allocating the memory for the field before knowing if the content can be
366 read from the event buffer is a mistake, as it might allocate a very large
367 amount of memory that will not be used.
368
369 So, alloc_and_memcpy ensures that it will only allocate memory for the field
370 if it can be read from the event buffer, avoiding allocating a memory that
371 will not be used.
372
373 @param[out] destination the destination buffer.
374 @param[in] length the amount of bytes to allocate and read from the buffer
375 (and to move forward).
376 @param[in] flags flags to pass to MySQL server my_malloc() function.
377 */
378 void alloc_and_memcpy(unsigned char **destination, size_t length, int flags);
379
380 /**
381 Allocates memory to a destination buffer, copies from the cursor to the
382 destination buffer using strncpy() and moves forward the cursor.
383
384 See comments on alloc_and_memcpy() for more details.
385
386 @param[out] destination the destination buffer.
387 @param[in] length the amount of bytes to allocate and read from the buffer
388 (and to forward).
389 @param[in] flags flags to pass to MySQL server my_malloc() function.
390 */
391 void alloc_and_strncpy(char **destination, size_t length, int flags);
392
393 /**
394 Reads string from cursor.
395
396 Reads in the following format:
397 1) Reads length stored on cursor first index. Moves cursor forward 1 byte.
398 2) Set destination pointer to the cursor. Moves cursor forward length bytes.
399
400 @param[out] destination the destination pointer.
401 @param[out] length the amount of bytes to allocate and read from the buffer
402 (and to move forward).
403 */
404 void read_str_at_most_255_bytes(const char **destination, uint8_t *length);
405
406 /**
407 Reads a packed value.
408
409 This function can move the cursor forward by 1, 3, 4 or 9 bytes depending on
410 the value to be returned.
411
412 @return the packed value.
413 */
414 uint64_t net_field_length_ll();
415
416 /**
417 Reads a transaction context data set.
418
419 @param[in] set_len length of the set object (and to move forward).
420 @param[out] set pointer to the set object to be filled.
421 */
422 void read_data_set(uint32_t set_len, std::list<const char *> *set);
423
424 /**
425 Reads a view change certification map.
426
427 @param[in] map_len the length of the certification info map (and to move
428 forward).
429 @param[out] map the certification info map to be filled.
430 */
431 void read_data_map(uint32_t map_len, std::map<std::string, std::string> *map);
432
433 /**
434 Copy a string into the destination buffer up to a max length.
435
436 @param[out] destination the destination buffer.
437 @param[in] max_length the max length to copy from the cursor.
438 @param[in] dest_length the max length supported by the destination buffer.
439 */
440 void strncpyz(char *destination, size_t max_length, size_t dest_length);
441
442 /**
443 Fills a vector with a sequence of bytes from the cursor.
444
445 @param[out] destination the vector be filled.
446 @param[in] length the amount of bytes to read from the cursor (and to move
447 forward).
448 */
449 void assign(std::vector<uint8_t> *destination, size_t length);
450
451 private:
452 /* The buffer with the serialized binary log event */
453 const char *m_buffer;
454 /* The cursor: a pointer to the current read position in the buffer */
455 const char *m_ptr;
456 /* The length of the buffer */
457 unsigned long long m_length;
458 /* The limit the reader shall respect when reading from the buffer */
459 unsigned long long m_limit;
460 /* The pointer to the current error message, or nullptr */
461 const char *m_error;
462
463 /**
464 Wrapper to le16toh to be used by read function.
465
466 @param[in] value the value to be converted.
467
468 @return the converted value.
469 */
470 uint16_t letoh(uint16_t value) { return le16toh(value); }
471
472 /**
473 Wrapper to le32toh to be used by read function.
474
475 @param[in] value the value to be converted.
476
477 @return the converted value.
478 */
479 int32_t letoh(int32_t value) { return le32toh(value); }
480
481 /**
482 Wrapper to le32toh to be used by read function.
483
484 @param[in] value the value to be converted.
485
486 @return the converted value.
487 */
488 uint32_t letoh(uint32_t value) { return le32toh(value); }
489
490 /**
491 Wrapper to le64toh to be used by read function.
492
493 @param[in] value the value to be converted.
494
495 @return the converted value.
496 */
497 int64_t letoh(int64_t value) { return le64toh(value); }
498
499 /**
500 Wrapper to le64toh to be used by read function.
501
502 @param[in] value the value to be converted.
503
504 @return the converted value.
505 */
506 uint64_t letoh(uint64_t value) { return le64toh(value); }
507};
508} // end namespace mysql::binlog::event
509
510namespace [[deprecated]] binary_log {
511using namespace mysql::binlog::event;
512} // namespace binary_log
513
514/// @}
515
516#endif // MYSQL_BINLOG_EVENT_EVENT_READER_H
Kerberos Client Authentication nullptr
Definition: auth_kerberos_client_plugin.cc:250
uint64_t le64toh(uint64_t x)
Converting a 64 bit integer from little-endian byte order to host byteorder.
Definition: byte_order_helpers.h:49
Event_reader class purpose is to avoid out-of-buffer reads when deserializing binary log events and i...
Definition: event_reader.h:73
bool has_error()
Returns if the Event_reader is in an error state or not.
Definition: event_reader.h:96
Event_reader(const char *buffer, unsigned long long length)
Event_reader constructor.
Definition: event_reader.h:83
void set_error(const char *error)
Sets Event_reader error state by setting the error message.
Definition: event_reader.cpp:29
uint64_t net_field_length_ll()
Reads a packed value.
Definition: event_reader.cpp:136
uint16_t letoh(uint16_t value)
Wrapper to le16toh to be used by read function.
Definition: event_reader.h:470
unsigned long long position()
Returns the current Event_reader cursor position in bytes.
Definition: event_reader.h:199
const char * get_error()
Returns the pointer to the error message.
Definition: event_reader.h:107
unsigned long long m_length
Definition: event_reader.h:457
void assign(std::vector< uint8_t > *destination, size_t length)
Fills a vector with a sequence of bytes from the cursor.
Definition: event_reader.cpp:213
int32_t letoh(int32_t value)
Wrapper to le32toh to be used by read function.
Definition: event_reader.h:479
void alloc_and_memcpy(unsigned char **destination, size_t length, int flags)
Allocates memory to a destination buffer, copies from the cursor to the destination buffer using memc...
Definition: event_reader.cpp:77
T read(unsigned char bytes=sizeof(T))
Copies a basic arithmetic type - uint8_t, [u]int16_t, [u]int32_t, [u]int64_t - from the buffer,...
Definition: event_reader.h:289
unsigned long long available_to_read()
Returns the amount of bytes still available to read from cursor position.
Definition: event_reader.h:208
unsigned long long length()
Returns the Event_reader buffer length.
Definition: event_reader.h:133
void read_data_set(uint32_t set_len, std::list< const char * > *set)
Reads a transaction context data set.
Definition: event_reader.cpp:159
const char * forward(unsigned long long bytes)
Moves the buffer position forward to a given relative position and returns the pointer to the buffer ...
Definition: event_reader.h:246
uint64_t letoh(uint64_t value)
Wrapper to le64toh to be used by read function.
Definition: event_reader.h:506
void set_length(unsigned long long length)
Sets Event_reader buffer length and limit.
Definition: event_reader.cpp:35
void memcpy(T destination, size_t length)
Copies from the cursor to an already existent (and allocated) buffer and moves forward the cursor.
Definition: event_reader.h:337
void strncpyz(char *destination, size_t max_length, size_t dest_length)
Copy a string into the destination buffer up to a max length.
Definition: event_reader.cpp:199
uint32_t letoh(uint32_t value)
Wrapper to le32toh to be used by read function.
Definition: event_reader.h:488
const char * m_buffer
Definition: event_reader.h:453
void read_data_map(uint32_t map_len, std::map< std::string, std::string > *map)
Reads a view change certification map.
Definition: event_reader.cpp:172
void alloc_and_strncpy(char **destination, size_t length, int flags)
Allocates memory to a destination buffer, copies from the cursor to the destination buffer using strn...
Definition: event_reader.cpp:96
T strndup(size_t length)
Returns a pointer to a new string which is a duplicate of the input string.
Definition: event_reader.h:315
unsigned long long m_limit
Definition: event_reader.h:459
void shrink_limit(unsigned long long bytes)
Shrinks the Event_reader buffer limit.
Definition: event_reader.cpp:45
bool can_read(unsigned long long bytes)
Returns if the Event_reader can read a given amount of bytes from cursor position.
Definition: event_reader.h:222
const char * buffer()
Returns the Event_reader buffer pointer.
Definition: event_reader.h:169
const char * ptr()
Returns a pointer to the Event_reader cursor (next position to be read by the Event_reader functions)...
Definition: event_reader.h:177
void read_str_at_most_255_bytes(const char **destination, uint8_t *length)
Reads string from cursor.
Definition: event_reader.cpp:116
int64_t letoh(int64_t value)
Wrapper to le64toh to be used by read function.
Definition: event_reader.h:497
T memcpy()
Copies a basic type - bool, char, int, long, double, etc - from the buffer, moves the cursor forward ...
Definition: event_reader.h:260
const char * go_to(unsigned long long position)
Moves cursor to a given absolute buffer position and returns the pointer to the cursor.
Definition: event_reader.cpp:66
const char * m_ptr
Definition: event_reader.h:455
const char * m_error
Definition: event_reader.h:461
#define PRINT_READER_STATUS(message)
Definition: event_reader.h:45
static int flags[50]
Definition: hp_test1.cc:39
The file contains functions to convert the byte encoding of integer values to and from little-endian ...
uint16_t le16toh(uint16_t x)
Converting a 16 bit integer from little-endian byte order to host byteorder.
Definition: byteorder.h:61
uint32_t le32toh(uint32_t x)
Converting a 32 bit integer from little-endian byte order to host byteorder.
Definition: byteorder.h:77
std::string str(const mysqlrouter::ConfigGenerator::Options::Endpoint &ep)
Definition: config_generator.cc:1065
Definition: base.h:74
The namespace contains classes representing events that can occur in a replication stream.
Definition: binlog_event.cpp:35
const char * bapi_strndup(const char *destination, size_t n)
This is a wrapper function, and returns a pointer to a new string which is a duplicate of the input s...
Definition: wrapper_functions.h:149
std::set< Key, Compare, ut::allocator< Key > > set
Specialization of set which uses ut_allocator.
Definition: ut0new.h:2881
std::map< Key, Value, Compare, ut::allocator< std::pair< const Key, Value > > > map
Specialization of map which uses ut_allocator.
Definition: ut0new.h:2891
Contains wrapper functions for memory allocation and deallocation.
#define BAPI_ASSERT(x)
Definition: wrapper_functions.h:60
#define BAPI_PRINT(name, params)
Definition: wrapper_functions.h:61