MySQL 8.4.3
Source Code Documentation
rpl_record.h
Go to the documentation of this file.
1/* Copyright (c) 2007, 2024, Oracle and/or its affiliates.
2
3 This program is free software; you can redistribute it and/or modify
4 it under the terms of the GNU General Public License, version 2.0,
5 as published by the Free Software Foundation.
6
7 This program is designed to work with certain software (including
8 but not limited to OpenSSL) that is licensed under separate terms,
9 as designated in a particular file or component or in included license
10 documentation. The authors of MySQL hereby grant you an additional
11 permission to link the program and your derivative works with the
12 separately licensed software that they have either included with
13 the program or referenced in the documentation.
14
15 This program is distributed in the hope that it will be useful,
16 but WITHOUT ANY WARRANTY; without even the implied warranty of
17 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
18 GNU General Public License, version 2.0, for more details.
19
20 You should have received a copy of the GNU General Public License
21 along with this program; if not, write to the Free Software
22 Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */
23
24#ifndef RPL_RECORD_H
25#define RPL_RECORD_H
26
27#include <stddef.h>
28#include <sys/types.h>
29
30#include "my_inttypes.h"
31
32class Relay_log_info;
33struct TABLE;
34
35struct MY_BITMAP;
36
37class table_def;
38class Bit_reader;
39
41
42#if defined(MYSQL_SERVER)
43size_t pack_row(TABLE *table, MY_BITMAP const *cols, uchar *row_data,
44 const uchar *data, enum_row_image_type row_image_type,
45 ulonglong value_options = 0);
46
47/**
48 Unpack a row image (either before-image or after-image) into @c
49 table->record[0].
50
51 The row is assumed to only consist of the fields for which the
52 corresponding bit in bitset @c column_image is set; the other parts
53 of the record are left alone.
54
55 If the replica table has more columns than the source table, then the
56 extra columns are not touched by this function. If the source table
57 has more columns than the replica table, then the position is moved to
58 after the extra columns, but the values are not used.
59
60 If the replica has a GIPK and the source does not, then the extra column
61 is not touched by this function. If the source table has a GIPK and the
62 replica does not, then the position is shifted forward by 1.
63
64 - The layout of a row is:
65
66 For WRITE_ROWS_EVENT:
67 +--------------+
68 | after-image |
69 +--------------+
70
71 For DELETE_ROWS_EVENT:
72 +--------------+
73 | before-image |
74 +--------------+
75
76 For UPDATE_ROWS_EVENT:
77 +--------------+-------------+
78 | before-image | after-image |
79 +--------------+-------------+
80
81 For PARTIAL_UPDATE_ROWS_EVENT:
82 +--------------+--------------+-------------+
83 | before-image | shared-image | after-image |
84 +--------------+--------------+-------------+
85
86 - Each of before-image and after-image has the following format:
87 +--------+-------+-------+ +-------+
88 | length | col_1 | col_2 | ... | col_N |
89 +--------+-------+-------+ +-------+
90 length is a 4-byte integer in little-endian format, equal to the
91 total length in bytes of col_1, col_2, ..., col_N.
92
93 - The shared-image has one of the following formats:
94 +-----------------+
95 | value_options=0 |
96 +-----------------+
97 or
98 +-----------------+--------------+
99 | value_options=1 | partial_bits |
100 +-----------------+--------------+
101 where:
102
103 - value_options is a bitmap, stored as an integer, in the format
104 of net_field_length. Currently only one bit is allowed:
105 1=PARTIAL_JSON_UPDATES (so therefore the integer is always 0 or
106 1, so in reality value_options is only one byte). When
107 PARTIAL_JSON_UPDATES=0, there is nothing else in the
108 shared-image. When PARTIAL_JSON_UPDATES=1, there is a
109 partial_bits field.
110
111 - partial_bits has one bit for each *JSON* column in the table
112 (regardless of whether it is included in the before-image and/or
113 after-image). The bit is 0 if the JSON update is stored as a
114 full document in the after-image, and 1 if the JSON update in
115 partial form in the after-image.
116
117 - Both when reading the before-image and when reading the
118 after-image it is necessary to know the partialness of JSON
119 columns: when reading the before-image, before looking up the
120 row in the table, we need to set the column in the table's
121 'read_set' (even if the column was not in the before-image), in
122 order to guarantee that the storage engine reads that column, so
123 that there is any base document that the diff can be applied
124 on. When reading the after-image, we need to know which columns
125 are partial so that we can correctly parse the data for that
126 column.
127
128 Therefore, when this function parses the before-image of a
129 PARTIAL_UPDATE_ROWS_LOG_EVENT, it reads both the before-image
130 and the shared-image, but leaves the read position after the
131 before-image. So when it parses the after-image of a
132 PARTIAL_UPDATE_ROWS_LOG_EVENT, the read position is at the
133 beginning of the shared-image, so it parses both the
134 shared-image and the after-image.
135
136 @param[in] rli Applier execution context
137
138 @param[in,out] table Table to unpack into
139
140 @param[in] source_column_count Number of columns that the source had
141 in its table
142
143 @param[in] row_data Beginning of the row image
144
145 @param[in] column_image Pointer to a bit vector where the N'th bit
146 is 0 for columns that are not included in the event, and 1 for
147 columns that are included in the event.
148
149 @param[out] row_image_end_p If this function returns successfully, it
150 sets row_image_end to point to the next byte after the row image
151 that it has read.
152
153 @param[in] event_end Pointer to the end of the event.
154
155 @param[in] row_image_type The type of row image that we are going to
156 read: WRITE_AI, UPDATE_BI, UPDATE_AI, or DELETE_BI.
157
158 @param[in] event_has_value_options true for PARTIAL_UPDATE_ROWS_EVENT,
159 false for UPDATE_ROWS_EVENT.
160
161 @param only_seek If true, this is a seek operation rather than a
162 read operation. It will only compute the row_image_end_p pointer,
163 and not read anything into the table and not apply any JSON diffs.
164 (This is used in HASH_SCAN, which (1) unpacks and hashes the
165 before-image for all rows in the event, (2)vscans the table, and
166 for each matching row it (3) unpacks the
167 after-image and applies on the table. In step (1) it needs to unpack
168 the after-image too, in order to move the read position forwards,
169 and then it should use only_seek=true. This is an optimization, but
170 more importantly, when the after-image contains partial JSON, the
171 partial JSON cannot be applied in step (1) since there is no JSON
172 document to apply it on.)
173
174 @returns false on success, true on error.
175 */
176bool unpack_row(Relay_log_info const *rli, TABLE *table,
177 uint const source_column_count, uchar const *const row_data,
178 MY_BITMAP const *column_image,
179 uchar const **const row_image_end_p,
180 uchar const *const event_end,
181 enum_row_image_type row_image_type,
182 bool event_has_value_options, bool only_seek);
183
184/**
185 Return a pointer within a row event's row data, to the data of the first
186 column that exists on the replica.
187
188 This skips the 'null bits' field, which precedes the column definitions in the
189 row image. In case a GIPK exists in the event but not in this replica's table
190 definition, it skips the GIPK too.
191
192 @param raw_data The data received from the source
193 @param column_image The column bitmap
194 @param column_count The number of column in the image bitmap
195 @param null_bits The bits that are null
196 @param tabledef The source table definition structure
197 @param source_has_gipk If the source table has a GIPK
198 @param replica_has_gipk If the replica table has a GIPK
199
200 @return const uchar* The adjusted point to the source data
201 */
203 const uchar *raw_data, MY_BITMAP const *column_image, size_t column_count,
204 Bit_reader &null_bits, table_def *tabledef, bool source_has_gipk,
205 bool replica_has_gipk);
206
207// Fill table's record[0] with default values.
208int prepare_record(TABLE *const table, const MY_BITMAP *cols, const bool check);
209#endif
210
211/**
212 Template base class of Bit_reader / Bit_writer.
213*/
214template <typename T, typename UT>
216 protected:
217 /// Pointer to beginning of buffer where bits are read or written.
219 /// Current position in buffer.
221
222 public:
223 /**
224 Construct a new Bit_stream (either reader or writer).
225 @param ptr Pointer where bits will be read or written.
226 */
227 Bit_stream_base(T *ptr) : m_ptr(ptr), m_current_bit(0) {}
228
229 /**
230 Set the buffer pointer.
231 @param ptr Pointer where bits will be read or written.
232 */
233 void set_ptr(T *ptr) { m_ptr = ptr; }
234 /**
235 Set the buffer pointer, using an unsigned datatype.
236 @param ptr Pointer where bits will be read or written.
237 */
238 void set_ptr(UT *ptr) { m_ptr = (T *)ptr; }
239
240 /// @return the current position.
241 uint tell() const { return m_current_bit; }
242
243 /**
244 Print all the bits before the current position to the debug trace.
245 @param str Descriptive text that will be prefixed before the bit string.
246 */
247 void dbug_print(const char *str [[maybe_unused]]) const;
248};
249
250/**
251 Auxiliary class to write a stream of bits to a memory location.
252
253 Call set() to write a bit and move the position one bit forward.
254*/
255class Bit_writer : public Bit_stream_base<char, uchar> {
256 public:
257 Bit_writer(char *ptr = nullptr) : Bit_stream_base<char, uchar>(ptr) {}
258 Bit_writer(uchar *ptr) : Bit_writer((char *)ptr) {}
259
260 /**
261 Write the next bit and move the write position one bit forward.
262 @param set_to_on If true, set the bit to 1, otherwise set it to 0.
263 */
264 void set(bool set_to_on) {
265 uint byte = m_current_bit / 8;
266 uint bit_within_byte = m_current_bit % 8;
268 if (bit_within_byte == 0)
269 m_ptr[byte] = set_to_on ? 1 : 0;
270 else if (set_to_on)
271 m_ptr[byte] |= 1 << bit_within_byte;
272 }
273};
274
275/**
276 Auxiliary class to read or write a stream of bits to a memory location.
277
278 Call get() to read a bit and move the position one bit forward.
279*/
280class Bit_reader : public Bit_stream_base<const char, const uchar> {
281 public:
282 Bit_reader(const char *ptr = nullptr)
283 : Bit_stream_base<const char, const uchar>(ptr) {}
284 Bit_reader(const uchar *ptr) : Bit_reader((const char *)ptr) {}
285
286 /**
287 Read the next bit and move the read position one bit forward.
288 @return true if the bit was 1, false if the bit was 0.
289 */
290 bool get() {
291 uint byte = m_current_bit / 8;
292 uint bit_within_byte = m_current_bit % 8;
294 return (m_ptr[byte] & (1 << bit_within_byte)) != 0;
295 }
296};
297
298#endif // ifdef RPL_RECORD_H
Auxiliary class to read or write a stream of bits to a memory location.
Definition: rpl_record.h:280
bool get()
Read the next bit and move the read position one bit forward.
Definition: rpl_record.h:290
Bit_reader(const uchar *ptr)
Definition: rpl_record.h:284
Bit_reader(const char *ptr=nullptr)
Definition: rpl_record.h:282
Template base class of Bit_reader / Bit_writer.
Definition: rpl_record.h:215
T * m_ptr
Pointer to beginning of buffer where bits are read or written.
Definition: rpl_record.h:218
uint m_current_bit
Current position in buffer.
Definition: rpl_record.h:220
uint tell() const
Definition: rpl_record.h:241
void set_ptr(UT *ptr)
Set the buffer pointer, using an unsigned datatype.
Definition: rpl_record.h:238
void set_ptr(T *ptr)
Set the buffer pointer.
Definition: rpl_record.h:233
void dbug_print(const char *str) const
Print all the bits before the current position to the debug trace.
Definition: rpl_record.cc:62
Bit_stream_base(T *ptr)
Construct a new Bit_stream (either reader or writer).
Definition: rpl_record.h:227
Auxiliary class to write a stream of bits to a memory location.
Definition: rpl_record.h:255
void set(bool set_to_on)
Write the next bit and move the write position one bit forward.
Definition: rpl_record.h:264
Bit_writer(uchar *ptr)
Definition: rpl_record.h:258
Bit_writer(char *ptr=nullptr)
Definition: rpl_record.h:257
Definition: rpl_rli.h:203
A table definition from the master.
Definition: rpl_utility.h:248
unsigned char byte
Blob class.
Definition: common.h:151
Some integer typedefs for easier portability.
unsigned long long int ulonglong
Definition: my_inttypes.h:56
unsigned char uchar
Definition: my_inttypes.h:52
std::string str(const mysqlrouter::ConfigGenerator::Options::Endpoint &ep)
Definition: config_generator.cc:1081
static PFS_engine_table_share_proxy table
Definition: pfs.cc:61
enum_row_image_type
Definition: rpl_record.h:40
const uchar * translate_beginning_of_raw_data(const uchar *raw_data, MY_BITMAP const *column_image, size_t column_count, Bit_reader &null_bits, table_def *tabledef, bool source_has_gipk, bool replica_has_gipk)
Return a pointer within a row event's row data, to the data of the first column that exists on the re...
Definition: rpl_record.cc:781
bool unpack_row(Relay_log_info const *rli, TABLE *table, uint const source_column_count, uchar const *const row_data, MY_BITMAP const *column_image, uchar const **const row_image_end_p, uchar const *const event_end, enum_row_image_type row_image_type, bool event_has_value_options, bool only_seek)
Unpack a row image (either before-image or after-image) into table->record[0].
Definition: rpl_record.cc:448
int prepare_record(TABLE *const table, const MY_BITMAP *cols, const bool check)
Fills table->record[0] with default values.
Definition: rpl_record.cc:811
size_t pack_row(TABLE *table, MY_BITMAP const *cols, uchar *row_data, const uchar *data, enum_row_image_type row_image_type, ulonglong value_options=0)
Pack a record of data for a table into a format suitable for the binary log.
Definition: rpl_record.cc:270
Definition: my_bitmap.h:43
Definition: table.h:1406