MySQL 9.1.0
Source Code Documentation
|
Class used for reading JSON values that are stored in the binary format. More...
#include <json_binary.h>
Public Types | |
enum | enum_type : uint8_t { OBJECT , ARRAY , STRING , INT , UINT , DOUBLE , LITERAL_NULL , LITERAL_TRUE , LITERAL_FALSE , OPAQUE , ERROR } |
Public Member Functions | |
EXPORT_JSON_FUNCTION bool | is_valid () const |
Does this value, and all of its members, represent a valid JSON value? More... | |
EXPORT_JSON_FUNCTION enum_type | type () const |
EXPORT_JSON_FUNCTION bool | large_format () const |
Does this value use the large storage format? More... | |
EXPORT_JSON_FUNCTION const char * | get_data () const |
Get a pointer to the beginning of the STRING or OPAQUE data represented by this instance. More... | |
EXPORT_JSON_FUNCTION uint32_t | get_data_length () const |
Get the length in bytes of the STRING or OPAQUE value represented by this instance. More... | |
EXPORT_JSON_FUNCTION uint32_t | get_container_length () const |
Get the length in bytes of the STRING or OPAQUE value represented by this instance. More... | |
EXPORT_JSON_FUNCTION const char * | get_container_data () const |
EXPORT_JSON_FUNCTION int64_t | get_int64 () const |
Get the value of an INT. More... | |
EXPORT_JSON_FUNCTION uint64_t | get_uint64 () const |
Get the value of a UINT. More... | |
EXPORT_JSON_FUNCTION double | get_double () const |
Get the value of a DOUBLE. More... | |
EXPORT_JSON_FUNCTION uint32_t | element_count () const |
Get the number of elements in an array, or the number of members in an object. More... | |
EXPORT_JSON_FUNCTION enum_field_types | field_type () const |
Get the MySQL field type of an opaque value. More... | |
EXPORT_JSON_FUNCTION Value | element (size_t pos) const |
Get the element at the specified position of a JSON array or a JSON object. More... | |
EXPORT_JSON_FUNCTION Value | key (size_t pos) const |
Get the key of the member stored at the specified position in a JSON object. More... | |
EXPORT_JSON_FUNCTION Value | lookup (std::string_view key) const |
Get the value associated with the specified key in a JSON object. More... | |
EXPORT_JSON_FUNCTION size_t | lookup_index (std::string_view key) const |
Get the index of the element with the specified key in a JSON object. More... | |
EXPORT_JSON_FUNCTION bool | is_backed_by (const String *str) const |
Is this binary value pointing to data that is contained in the specified string. More... | |
EXPORT_JSON_FUNCTION bool | raw_binary (const JsonSerializationErrorHandler &error_handler, String *buf) const |
Copy the binary representation of this value into a buffer, replacing the contents of the receiving buffer. More... | |
EXPORT_JSON_FUNCTION bool | get_free_space (const JsonSerializationErrorHandler &error_handler, size_t *space) const |
Get the amount of unused space in the binary representation of this value. More... | |
bool | update_in_shadow (const Field_json *field, size_t pos, Json_wrapper *new_value, size_t data_offset, size_t data_length, const char *original, char *destination, bool *changed) const |
Update a value in an array or object. More... | |
bool | remove_in_shadow (const Field_json *field, size_t pos, const char *original, char *destination) const |
Remove a value from an array or object. More... | |
EXPORT_JSON_FUNCTION bool | has_space (size_t pos, size_t needed, size_t *offset) const |
Does this array or object have enough space to replace the value at the given position with another value of a given size? More... | |
EXPORT_JSON_FUNCTION | Value (enum_type t) |
Constructor for values that represent literals or errors. More... | |
EXPORT_JSON_FUNCTION | Value (enum_type t, int64_t val) |
Constructor for values that represent ints or uints. More... | |
EXPORT_JSON_FUNCTION | Value (double val) |
Constructor for values that represent doubles. More... | |
EXPORT_JSON_FUNCTION | Value (const char *data, uint32_t len) |
Constructor for values that represent strings. More... | |
EXPORT_JSON_FUNCTION | Value (enum_type t, const char *data, uint32_t bytes, uint32_t element_count, bool large) |
Constructor for values that represent arrays or objects. More... | |
EXPORT_JSON_FUNCTION | Value (enum_field_types ft, const char *data, uint32_t len) |
Constructor for values that represent opaque data. More... | |
EXPORT_JSON_FUNCTION | Value () |
Empty constructor. More... | |
EXPORT_JSON_FUNCTION bool | is_array () const |
Is this value an array? More... | |
EXPORT_JSON_FUNCTION bool | is_object () const |
Is this value an object? More... | |
EXPORT_JSON_FUNCTION bool | to_std_string (std::string *buffer, const JsonErrorHandler &depth_handler) const |
Format the JSON value to an external JSON string in buffer in the format of ISO/IEC 10646. More... | |
EXPORT_JSON_FUNCTION bool | to_pretty_std_string (std::string *buffer, const JsonErrorHandler &depth_handler) const |
Format the JSON value to an external JSON string in buffer in the format of ISO/IEC 10646. More... | |
int | eq (const Value &val) const |
Compare two Values. More... | |
EXPORT_JSON_FUNCTION size_t | key_entry_offset (size_t pos) const |
Get the offset of the key entry that describes the key of the member at a given position in this object. More... | |
EXPORT_JSON_FUNCTION size_t | value_entry_offset (size_t pos) const |
Get the offset of the value entry that describes the element at a given position in this array or object. More... | |
Private Member Functions | |
bool | first_value_offset (size_t *offset) const |
Find the lowest possible offset where a value can be located inside this array or object. More... | |
bool | element_offsets (size_t pos, size_t *start, size_t *end, bool *inlined) const |
Find the start offset and the end offset of the specified element. More... | |
Private Attributes | |
union { | |
const char * m_data | |
Pointer to the start of the binary representation of the value. More... | |
int64_t m_int_value | |
The value if the type is INT or UINT. More... | |
double m_double_value | |
The value if the type is DOUBLE. More... | |
}; | |
uint32_t | m_element_count |
Element count for arrays and objects. More... | |
uint32_t | m_length |
The full length (in bytes) of the binary representation of an array or object, or the length of a string or opaque value. More... | |
enum_field_types | m_field_type |
The MySQL field type of the value, in case the type of the value is OPAQUE. More... | |
enum_type | m_type |
The JSON type of the value. More... | |
bool | m_large |
True if an array or an object uses the large storage format with 4 byte offsets instead of 2 byte offsets. More... | |
Class used for reading JSON values that are stored in the binary format.
Values are parsed lazily, so that only the parts of the value that are interesting to the caller, are read. Array elements can be looked up in constant time using the element() function. Object members can be looked up in O(log n) time using the lookup() function.
enum json_binary::Value::enum_type : uint8_t |
|
inlineexplicit |
Constructor for values that represent literals or errors.
|
inlineexplicit |
Constructor for values that represent ints or uints.
|
inlineexplicit |
Constructor for values that represent doubles.
|
inline |
Constructor for values that represent strings.
|
inline |
Constructor for values that represent arrays or objects.
t | type |
data | pointer to the start of the binary representation |
bytes | the number of bytes in the binary representation of the value |
element_count | the number of elements or members in the value |
large | true if the value should be stored in the large storage format with 4 byte offsets instead of 2 byte offsets |
|
inline |
Constructor for values that represent opaque data.
|
inline |
Empty constructor.
Produces a value that represents an error condition.
Value json_binary::Value::element | ( | size_t | pos | ) | const |
Get the element at the specified position of a JSON array or a JSON object.
When called on a JSON object, it returns the value associated with the key returned by key(pos).
pos | the index of the element |
|
inline |
Get the number of elements in an array, or the number of members in an object.
|
private |
Find the start offset and the end offset of the specified element.
[in] | pos | which element to check |
[out] | start | the start offset of the value |
[out] | end | the end offset of the value (exclusive) |
[out] | inlined | set to true if the specified element is inlined |
int json_binary::Value::eq | ( | const Value & | val | ) | const |
Compare two Values.
Check whether two binary JSON scalars are equal.
This function is used by multi-valued index updating code. Unlike JSON comparator implemented in server, this code doesn't treat numeric types as the same, e.g. int 1 and uint 1 won't be treated as equal. This is fine as the mv index updating code compares old and new values of the same typed array field, i.e. all values being compared have the same type.
Since MV index doesn't support indexing of arrays/objects in arrays, these two aren't supported and cause assert.
|
inline |
Get the MySQL field type of an opaque value.
Identifies the type of the value stored in the data portion of an opaque value.
|
private |
Find the lowest possible offset where a value can be located inside this array or object.
[out] | offset | the lowest offset where a value can be located |
|
inline |
|
inline |
Get the length in bytes of the STRING or OPAQUE value represented by this instance.
|
inline |
Get a pointer to the beginning of the STRING or OPAQUE data represented by this instance.
|
inline |
Get the length in bytes of the STRING or OPAQUE value represented by this instance.
|
inline |
Get the value of a DOUBLE.
bool json_binary::Value::get_free_space | ( | const JsonSerializationErrorHandler & | error_handler, |
size_t * | space | ||
) | const |
Get the amount of unused space in the binary representation of this value.
[out] | error_handler | the handler that is invoked if an error occurs |
[out] | space | the amount of free space |
|
inline |
Get the value of an INT.
|
inline |
Get the value of a UINT.
bool json_binary::Value::has_space | ( | size_t | pos, |
size_t | needed, | ||
size_t * | offset | ||
) | const |
Does this array or object have enough space to replace the value at the given position with another value of a given size?
[in] | pos | the position in the array or object |
[in] | needed | the number of bytes needed for the new value |
[out] | offset | if true is returned, this value is set to an offset relative to the start of the array or object, which tells where the replacement value should be stored |
|
inline |
Is this value an array?
bool json_binary::Value::is_backed_by | ( | const String * | str | ) | const |
Is this binary value pointing to data that is contained in the specified string.
str | a string with binary data |
true | if the string contains data pointed to from this object |
false | otherwise |
|
inline |
Is this value an object?
bool json_binary::Value::is_valid | ( | void | ) | const |
Does this value, and all of its members, represent a valid JSON value?
Value json_binary::Value::key | ( | size_t | pos | ) | const |
Get the key of the member stored at the specified position in a JSON object.
pos | the index of the member |
size_t json_binary::Value::key_entry_offset | ( | size_t | pos | ) | const |
Get the offset of the key entry that describes the key of the member at a given position in this object.
pos | the position of the member |
|
inline |
Does this value use the large storage format?
Value json_binary::Value::lookup | ( | std::string_view | key | ) | const |
Get the value associated with the specified key in a JSON object.
[in] | key | the key to look up |
size_t json_binary::Value::lookup_index | ( | std::string_view | key | ) | const |
Get the index of the element with the specified key in a JSON object.
[in] | key | the key to look up |
element_count()
if the key is not found bool json_binary::Value::raw_binary | ( | const JsonSerializationErrorHandler & | error_handler, |
String * | buf | ||
) | const |
Copy the binary representation of this value into a buffer, replacing the contents of the receiving buffer.
error_handler | a handler that is invoked if an error occurs |
buf | the receiving buffer |
bool json_binary::Value::remove_in_shadow | ( | const Field_json * | field, |
size_t | pos, | ||
const char * | original, | ||
char * | destination | ||
) | const |
Remove a value from an array or object.
The updated JSON document is written to a shadow copy. The original document is left unchanged, unless the shadow copy is actually a pointer to the array backing this Value object. It is assumed that the shadow copy is at least as big as the original document, and that there is enough space at the given position to hold the new value.
Typically, if a document is modified multiple times in a single update statement, the first invocation of remove_in_shadow() will have a Value object that points into the binary data in the Field, and write to a separate destination buffer. Subsequent updates of the document will have a Value object that points to the partially updated value in the destination buffer, and write the new modifications to the same buffer.
All changes made to the binary value are recorded as binary diffs using TABLE::add_binary_diff().
field | the column that is updated |
pos | the element to remove |
original | pointer to the start of the JSON document |
destination | pointer to the shadow copy of the JSON document (it could be the same as original, in which case the original document will be modified) |
Take the JSON document { "a": "x", "b": "y", "c": "z" }, whose serialized representation looks like the following:
0x00 - type: JSONB_TYPE_SMALL_OBJECT 0x03 - number of elements (low byte) 0x00 - number of elements (high byte) 0x22 - number of bytes (low byte) 0x00 - number of bytes (high byte) 0x19 - offset of key "a" (high byte) 0x00 - offset of key "a" (low byte) 0x01 - length of key "a" (high byte) 0x00 - length of key "a" (low byte) 0x1a - offset of key "b" (high byte) 0x00 - offset of key "b" (low byte) 0x01 - length of key "b" (high byte) 0x00 - length of key "b" (low byte) 0x1b - offset of key "c" (high byte) 0x00 - offset of key "c" (low byte) 0x01 - length of key "c" (high byte) 0x00 - length of key "c" (low byte) 0x0c - type of value "a": JSONB_TYPE_STRING 0x1c - offset of value "a" (high byte) 0x00 - offset of value "a" (low byte) 0x0c - type of value "b": JSONB_TYPE_STRING 0x1e - offset of value "b" (high byte) 0x00 - offset of value "b" (low byte) 0x0c - type of value "c": JSONB_TYPE_STRING 0x20 - offset of value "c" (high byte) 0x00 - offset of value "c" (low byte) 0x61 - first key ('a') 0x62 - second key ('b') 0x63 - third key ('c') 0x01 - length of value "a" 0x78 - contents of value "a" ('x') 0x01 - length of value "b" 0x79 - contents of value "b" ('y') 0x01 - length of value "c" 0x7a - contents of value "c" ('z')
We remove the member with name 'b' from the document, using a statement such as:
UPDATE t SET j = JSON_REMOVE(j, '$.b')
This function will then remove the element by moving the key entries and value entries that follow the removed member so that they overwrite the existing entries, and the element count is decremented.
The resulting binary document will look like this:
0x00 - type: JSONB_TYPE_SMALL_OBJECT CHANGED 0x02 - number of elements (low byte) 0x00 - number of elements (high byte) 0x22 - number of bytes (low byte) 0x00 - number of bytes (high byte) 0x19 - offset of key "a" (high byte) 0x00 - offset of key "a" (low byte) 0x01 - length of key "a" (high byte) 0x00 - length of key "a" (low byte) CHANGED 0x1b - offset of key "c" (high byte) CHANGED 0x00 - offset of key "c" (low byte) CHANGED 0x01 - length of key "c" (high byte) CHANGED 0x00 - length of key "c" (low byte) CHANGED 0x0c - type of value "a": JSONB_TYPE_STRING CHANGED 0x1c - offset of value "a" (high byte) CHANGED 0x00 - offset of value "a" (low byte) CHANGED 0x0c - type of value "c": JSONB_TYPE_STRING CHANGED 0x20 - offset of value "c" (high byte) CHANGED 0x00 - offset of value "c" (low byte) (free) 0x00 (free) 0x0c (free) 0x1e (free) 0x00 (free) 0x0c (free) 0x20 (free) 0x00 0x61 - first key ('a') (free) 0x62 0x63 - third key ('c') 0x01 - length of value "a" 0x78 - contents of value "a" ('x') (free) 0x01 (free) 0x79 0x01 - length of value "c" 0x7a - contents of value "c" ('z')
Two binary diffs will be created. One diff changes the element count, and one diff changes the key and value entries.
bool json_binary::Value::to_pretty_std_string | ( | std::string * | buffer, |
const JsonErrorHandler & | depth_handler | ||
) | const |
Format the JSON value to an external JSON string in buffer in the format of ISO/IEC 10646.
Add newlines and indentation for readability.
[in,out] | buffer | the buffer that receives the formatted string (the string is appended, so make sure the length is set correctly before calling) |
[in] | depth_handler | Pointer to a function that should handle error occurred when depth is exceeded. |
false | on success |
true | on error |
bool json_binary::Value::to_std_string | ( | std::string * | buffer, |
const JsonErrorHandler & | depth_handler | ||
) | const |
Format the JSON value to an external JSON string in buffer in the format of ISO/IEC 10646.
[in,out] | buffer | the formatted string is appended, so make sure the length is set correctly before calling |
[in] | depth_handler | Pointer to a function that should handle error occurred when depth is exceeded. |
|
inline |
bool json_binary::Value::update_in_shadow | ( | const Field_json * | field, |
size_t | pos, | ||
Json_wrapper * | new_value, | ||
size_t | data_offset, | ||
size_t | data_length, | ||
const char * | original, | ||
char * | destination, | ||
bool * | changed | ||
) | const |
Update a value in an array or object.
The updated value is written to a shadow copy. The original array or object is left unchanged, unless the shadow copy is actually a pointer to the array backing this Value object. It is assumed that the shadow copy is at least as big as the original document, and that there is enough space at the given position to hold the new value.
Typically, if a document is modified multiple times in a single update statement, the first invocation of update_in_shadow() will have a Value object that points into the binary data in the Field, and write to a separate destination buffer. Subsequent updates of the document will have a Value object that points to the partially updated value in the destination buffer, and write the new modifications to the same buffer.
All changes made to the binary value are recorded as binary diffs using TABLE::add_binary_diff().
field | the column that is updated | |
pos | the element to update | |
new_value | the new value of the element | |
data_offset | where to write the value (offset relative to the beginning of the array or object, obtained with has_space) or zero if the value can be inlined | |
data_length | the length of the new value in bytes or zero if the value can be inlined | |
original | pointer to the start of the JSON document | |
destination | pointer to the shadow copy of the JSON document (it could be the same as original, in which case the original document will be modified) | |
[out] | changed | gets set to true if a change was made to the document, or to false if this operation was a no-op |
Given the JSON document [ "abc", "def" ], which is serialized like this in a JSON column:
0x02 - type: small JSON array 0x02 - number of elements (low byte) 0x00 - number of elements (high byte) 0x12 - number of bytes (low byte) 0x00 - number of bytes (high byte) 0x0C - type of element 0 (string) 0x0A - offset of element 0 (low byte) 0x00 - offset of element 0 (high byte) 0x0C - type of element 1 (string) 0x0E - offset of element 1 (low byte) 0x00 - offset of element 1 (high byte) 0x03 - length of element 0 'a' 'b' - content of element 0 'c' 0x03 - length of element 1 'd' 'e' - content of element 1 'f'
Let's change element 0 from "abc" to "XY" using the following statement:
UPDATE t SET j = JSON_SET(j, '$[0]', 'XY')
Since we're replacing one string with a shorter one, we can just overwrite the length byte with the new length, and the beginning of the original string data. Since the original string "abc" is longer than the new string "XY", we'll have a free byte at the end of the string. This byte is left as is ('c'). The resulting binary representation looks like this:
0x02 - type: small JSON array 0x02 - number of elements (low byte) 0x00 - number of elements (high byte) 0x12 - number of bytes (low byte) 0x00 - number of bytes (high byte) 0x0C - type of element 0 (string) 0x0A - offset of element 0 (low byte) 0x00 - offset of element 0 (high byte) 0x0C - type of element 1 (string) 0x0E - offset of element 1 (low byte) 0x00 - offset of element 1 (high byte) CHANGED 0x02 - length of element 0 CHANGED 'X' CHANGED 'Y' - content of element 0 (free) 'c' 0x03 - length of element 1 'd' 'e' - content of element 1 'f'
This change will be represented as one binary diff that covers the three changed bytes.
Let's now change element 1 from "def" to "XYZW":
UPDATE t SET j = JSON_SET(j, '$[1]', 'XYZW')
Since the new string is one byte longer than the original string, we cannot simply overwrite the old one. But we can reuse the free byte from the previous update, which is immediately preceding the original value.
To make use of this, we need to change the offset of element 1 to point to the free byte. Then we can overwrite the free byte and the original string data with the new length and string contents. Resulting binary representation:
0x02 - type: small JSON array 0x02 - number of elements (low byte) 0x00 - number of elements (high byte) 0x12 - number of bytes (low byte) 0x00 - number of bytes (high byte) 0x0C - type of element 0 (string) 0x0A - offset of element 0 (low byte) 0x00 - offset of element 0 (high byte) 0x0C - type of element 1 (string) CHANGED 0x0D - offset of element 1 (low byte) 0x00 - offset of element 1 (high byte) 0x02 - length of element 0 'X' - content of element 0 'Y' - content of element 0 CHANGED 0x04 - length of element 1 CHANGED 'X' CHANGED 'Y' CHANGED 'Z' - content of element 1 CHANGED 'W'
This change will be represented as two binary diffs. One diff for changing the offset, and one for changing the contents of the string.
Then let's replace the string in element 1 with a small number:
UPDATE t SET j = JSON_SET(j, '$[1]', 456)
This will change the type of element 1 from string to int16. Such small numbers are inlined in the value entry, where we normally store the offset of the value. The offset section of the value entry is therefore changed to hold the number 456. The length and contents of the original value ("XYZW") are not touched, but they are now unused and free to be reused. Resulting binary representation:
0x02 - type: small JSON array 0x02 - number of elements (low byte) 0x00 - number of elements (high byte) 0x12 - number of bytes (low byte) 0x00 - number of bytes (high byte) 0x0C - type of element 0 (string) 0x0A - offset of element 0 (low byte) 0x00 - offset of element 0 (high byte) CHANGED 0x05 - type of element 1 (int16) CHANGED 0xC8 - value of element 1 (low byte) CHANGED 0x01 - value of element 1 (high byte) 0x02 - length of element 0 'X' - content of element 0 'Y' - content of element 0 (free) 0x04 - length of element 1 (free) 'X' (free) 'Y' (free) 'Z' - content of element 1 (free) 'W'
The change is represented as one binary diff that changes the value entry (type and inlined value).
size_t json_binary::Value::value_entry_offset | ( | size_t | pos | ) | const |
Get the offset of the value entry that describes the element at a given position in this array or object.
pos | the position of the element |
union { ... } json_binary::Value::@37 |
const char* json_binary::Value::m_data |
Pointer to the start of the binary representation of the value.
Only used by STRING, OPAQUE, OBJECT and ARRAY.
The memory pointed to by this member is not owned by this Value object. Callers that create Value objects must make sure that the memory is not freed as long as the Value object is alive.
double json_binary::Value::m_double_value |
The value if the type is DOUBLE.
|
private |
Element count for arrays and objects.
Unused for other types.
|
private |
The MySQL field type of the value, in case the type of the value is OPAQUE.
Otherwise, it is unused.
int64_t json_binary::Value::m_int_value |
The value if the type is INT or UINT.
|
private |
True if an array or an object uses the large storage format with 4 byte offsets instead of 2 byte offsets.
|
private |
The full length (in bytes) of the binary representation of an array or object, or the length of a string or opaque value.
Unused for other types.
|
private |
The JSON type of the value.