MySQL  8.0.17
Source Code Documentation
json_binary::Value Class Reference

Class used for reading JSON values that are stored in the binary format. More...

#include <json_binary.h>

Public Types

enum  enum_type : uint8 {
  OBJECT, ARRAY, STRING, INT,
  UINT, DOUBLE, LITERAL_NULL, LITERAL_TRUE,
  LITERAL_FALSE, OPAQUE, ERROR
}
 

Public Member Functions

bool is_valid () const
 Does this value, and all of its members, represent a valid JSON value? More...
 
enum_type type () const
 
bool large_format () const
 Does this value use the large storage format? More...
 
const char * get_data () const
 Get a pointer to the beginning of the STRING or OPAQUE data represented by this instance. More...
 
uint32 get_data_length () const
 Get the length in bytes of the STRING or OPAQUE value represented by this instance. More...
 
int64 get_int64 () const
 Get the value of an INT. More...
 
uint64 get_uint64 () const
 Get the value of a UINT. More...
 
double get_double () const
 Get the value of a DOUBLE. More...
 
uint32 element_count () const
 Get the number of elements in an array, or the number of members in an object. More...
 
enum_field_types field_type () const
 Get the MySQL field type of an opaque value. More...
 
Value element (size_t pos) const
 Get the element at the specified position of a JSON array or a JSON object. More...
 
Value key (size_t pos) const
 Get the key of the member stored at the specified position in a JSON object. More...
 
Value lookup (const char *key, size_t length) const
 Get the value associated with the specified key in a JSON object. More...
 
Value lookup (const std::string &key) const
 
size_t lookup_index (const char *key, size_t length) const
 Get the index of the element with the specified key in a JSON object. More...
 
size_t lookup_index (const std::string &key) const
 
bool is_backed_by (const String *str) const
 Is this binary value pointing to data that is contained in the specified string. More...
 
bool raw_binary (const THD *thd, String *buf) const
 Copy the binary representation of this value into a buffer, replacing the contents of the receiving buffer. More...
 
bool get_free_space (const THD *thd, size_t *space) const
 Get the amount of unused space in the binary representation of this value. More...
 
bool has_space (size_t pos, size_t needed, size_t *offset) const
 Does this array or object have enough space to replace the value at the given position with another value of a given size? More...
 
bool update_in_shadow (const Field_json *field, size_t pos, Json_wrapper *new_value, size_t data_offset, size_t data_length, const char *original, char *destination, bool *changed) const
 Update a value in an array or object. More...
 
bool remove_in_shadow (const Field_json *field, size_t pos, const char *original, char *destination) const
 Remove a value from an array or object. More...
 
 Value (enum_type t)
 Constructor for values that represent literals or errors. More...
 
 Value (enum_type t, int64 val)
 Constructor for values that represent ints or uints. More...
 
 Value (double val)
 Constructor for values that represent doubles. More...
 
 Value (const char *data, uint32 len)
 Constructor for values that represent strings. More...
 
 Value (enum_type t, const char *data, uint32 bytes, uint32 element_count, bool large)
 Constructor for values that represent arrays or objects. More...
 
 Value (enum_field_types ft, const char *data, uint32 len)
 Constructor for values that represent opaque data. More...
 
 Value ()
 Empty constructor. More...
 
bool is_array () const
 Is this value an array? More...
 
bool is_object () const
 Is this value an object? More...
 
int eq (const Value &val) const
 Compare two Values. More...
 

Private Member Functions

size_t key_entry_offset (size_t pos) const
 Get the offset of the key entry that describes the key of the member at a given position in this object. More...
 
size_t value_entry_offset (size_t pos) const
 Get the offset of the value entry that describes the element at a given position in this array or object. More...
 
bool first_value_offset (size_t *offset) const
 Find the lowest possible offset where a value can be located inside this array or object. More...
 
bool element_offsets (size_t pos, size_t *start, size_t *end, bool *inlined) const
 Find the start offset and the end offset of the specified element. More...
 

Private Attributes

union {
   const char *   m_data
 Pointer to the start of the binary representation of the value. More...
 
   int64   m_int_value
 The value if the type is INT or UINT. More...
 
   double   m_double_value
 The value if the type is DOUBLE. More...
 
}; 
 
uint32 m_element_count
 Element count for arrays and objects. More...
 
uint32 m_length
 The full length (in bytes) of the binary representation of an array or object, or the length of a string or opaque value. More...
 
enum_field_types m_field_type
 The MySQL field type of the value, in case the type of the value is OPAQUE. More...
 
enum_type m_type
 The JSON type of the value. More...
 
bool m_large
 True if an array or an object uses the large storage format with 4 byte offsets instead of 2 byte offsets. More...
 

Detailed Description

Class used for reading JSON values that are stored in the binary format.

Values are parsed lazily, so that only the parts of the value that are interesting to the caller, are read. Array elements can be looked up in constant time using the element() function. Object members can be looked up in O(log n) time using the lookup() function.

Member Enumeration Documentation

◆ enum_type

Enumerator
OBJECT 
ARRAY 
STRING 
INT 
UINT 
DOUBLE 
LITERAL_NULL 
LITERAL_TRUE 
LITERAL_FALSE 
OPAQUE 
ERROR 

Constructor & Destructor Documentation

◆ Value() [1/7]

json_binary::Value::Value ( enum_type  t)
inlineexplicit

Constructor for values that represent literals or errors.

◆ Value() [2/7]

json_binary::Value::Value ( enum_type  t,
int64  val 
)
inlineexplicit

Constructor for values that represent ints or uints.

◆ Value() [3/7]

json_binary::Value::Value ( double  val)
inlineexplicit

Constructor for values that represent doubles.

◆ Value() [4/7]

json_binary::Value::Value ( const char *  data,
uint32  len 
)
inline

Constructor for values that represent strings.

◆ Value() [5/7]

json_binary::Value::Value ( enum_type  t,
const char *  data,
uint32  bytes,
uint32  element_count,
bool  large 
)
inline

Constructor for values that represent arrays or objects.

Parameters
ttype
datapointer to the start of the binary representation
bytesthe number of bytes in the binary representation of the value
element_countthe number of elements or members in the value
largetrue if the value should be stored in the large storage format with 4 byte offsets instead of 2 byte offsets

◆ Value() [6/7]

json_binary::Value::Value ( enum_field_types  ft,
const char *  data,
uint32  len 
)
inline

Constructor for values that represent opaque data.

◆ Value() [7/7]

json_binary::Value::Value ( )
inline

Empty constructor.

Produces a value that represents an error condition.

Member Function Documentation

◆ element()

Value json_binary::Value::element ( size_t  pos) const

Get the element at the specified position of a JSON array or a JSON object.

When called on a JSON object, it returns the value associated with the key returned by key(pos).

Parameters
posthe index of the element
Returns
a value representing the specified element, or a value where type() returns ERROR if pos does not point to an element

◆ element_count()

uint32 json_binary::Value::element_count ( ) const
inline

Get the number of elements in an array, or the number of members in an object.

◆ element_offsets()

bool json_binary::Value::element_offsets ( size_t  pos,
size_t *  start,
size_t *  end,
bool inlined 
) const
private

Find the start offset and the end offset of the specified element.

Parameters
[in]poswhich element to check
[out]startthe start offset of the value
[out]endthe end offset of the value (exclusive)
[out]inlinedset to true if the specified element is inlined
Returns
true if the offsets cannot be determined, false if successful

◆ eq()

int json_binary::Value::eq ( const Value val) const

Compare two Values.

Check whether two binary JSON scalars are equal.

Note
This function is limited to scalars only, for objects/arrays it asserts. The main purpose is to separate old/new scalar values for updates on multi-valued indexes.
Returns
-1 this < val 0 this == val 1 this > val

This function is used by multi-valued index updating code. Unlike JSON comparator implemented in server, this code doesn't treat numeric types as the same, e.g. int 1 and uint 1 won't be treated as equal. This is fine as the mv index updating code compares old and new values of the same typed array field, i.e. all values being compared have the same type.

Since MV index doesn't support indexing of arrays/objects in arrays, these two aren't supported and cause assert.

◆ field_type()

enum_field_types json_binary::Value::field_type ( ) const
inline

Get the MySQL field type of an opaque value.

Identifies the type of the value stored in the data portion of an opaque value.

◆ first_value_offset()

bool json_binary::Value::first_value_offset ( size_t *  offset) const
private

Find the lowest possible offset where a value can be located inside this array or object.

Parameters
[out]offsetthe lowest offset where a value can be located
Returns
false on success, true on error

◆ get_data()

const char* json_binary::Value::get_data ( ) const
inline

Get a pointer to the beginning of the STRING or OPAQUE data represented by this instance.

◆ get_data_length()

uint32 json_binary::Value::get_data_length ( ) const
inline

Get the length in bytes of the STRING or OPAQUE value represented by this instance.

◆ get_double()

double json_binary::Value::get_double ( ) const
inline

Get the value of a DOUBLE.

◆ get_free_space()

bool json_binary::Value::get_free_space ( const THD thd,
size_t *  space 
) const

Get the amount of unused space in the binary representation of this value.

Parameters
thdTHD handle
[out]spacethe amount of free space
Returns
false on success, true on error

◆ get_int64()

int64 json_binary::Value::get_int64 ( ) const
inline

Get the value of an INT.

◆ get_uint64()

uint64 json_binary::Value::get_uint64 ( ) const
inline

Get the value of a UINT.

◆ has_space()

bool json_binary::Value::has_space ( size_t  pos,
size_t  needed,
size_t *  offset 
) const

Does this array or object have enough space to replace the value at the given position with another value of a given size?

Parameters
[in]posthe position in the array or object
[in]neededthe number of bytes needed for the new value
[out]offsetif true is returned, this value is set to an offset relative to the start of the array or object, which tells where the replacement value should be stored
Returns
true if there is enough space, false otherwise

◆ is_array()

bool json_binary::Value::is_array ( ) const
inline

Is this value an array?

◆ is_backed_by()

bool json_binary::Value::is_backed_by ( const String str) const

Is this binary value pointing to data that is contained in the specified string.

Parameters
stra string with binary data
Return values
trueif the string contains data pointed to from this object
falseotherwise

◆ is_object()

bool json_binary::Value::is_object ( ) const
inline

Is this value an object?

◆ is_valid()

bool json_binary::Value::is_valid ( ) const

Does this value, and all of its members, represent a valid JSON value?

◆ key()

Value json_binary::Value::key ( size_t  pos) const

Get the key of the member stored at the specified position in a JSON object.

Parameters
posthe index of the member
Returns
the key of the specified member, or a value where type() returns ERROR if pos does not point to a member

◆ key_entry_offset()

size_t json_binary::Value::key_entry_offset ( size_t  pos) const
inlineprivate

Get the offset of the key entry that describes the key of the member at a given position in this object.

Parameters
posthe position of the member
Returns
the offset of the key entry, relative to the start of the object

◆ large_format()

bool json_binary::Value::large_format ( ) const
inline

Does this value use the large storage format?

◆ lookup() [1/2]

Value json_binary::Value::lookup ( const char *  key,
size_t  length 
) const

Get the value associated with the specified key in a JSON object.

Parameters
[in]keythe key to look up
[in]lengththe length of the key
Returns
the value associated with the key, if there is one. otherwise, returns ERROR

◆ lookup() [2/2]

Value json_binary::Value::lookup ( const std::string &  key) const
inline

◆ lookup_index() [1/2]

size_t json_binary::Value::lookup_index ( const char *  key,
size_t  length 
) const

Get the index of the element with the specified key in a JSON object.

Parameters
[in]keythe key to look up
[in]lengththe length of the key
Returns
the index if the key is found, or element_count() if the key is not found

◆ lookup_index() [2/2]

size_t json_binary::Value::lookup_index ( const std::string &  key) const
inline

◆ raw_binary()

bool json_binary::Value::raw_binary ( const THD thd,
String buf 
) const

Copy the binary representation of this value into a buffer, replacing the contents of the receiving buffer.

Parameters
thdTHD handle
bufthe receiving buffer
Returns
false on success, true otherwise

◆ remove_in_shadow()

bool json_binary::Value::remove_in_shadow ( const Field_json field,
size_t  pos,
const char *  original,
char *  destination 
) const

Remove a value from an array or object.

The updated JSON document is written to a shadow copy. The original document is left unchanged, unless the shadow copy is actually a pointer to the array backing this Value object. It is assumed that the shadow copy is at least as big as the original document, and that there is enough space at the given position to hold the new value.

Typically, if a document is modified multiple times in a single update statement, the first invocation of remove_in_shadow() will have a Value object that points into the binary data in the Field, and write to a separate destination buffer. Subsequent updates of the document will have a Value object that points to the partially updated value in the destination buffer, and write the new modifications to the same buffer.

All changes made to the binary value are recorded as binary diffs using TABLE::add_binary_diff().

Parameters
fieldthe column that is updated
posthe element to remove
originalpointer to the start of the JSON document
destinationpointer to the shadow copy of the JSON document (it could be the same as original, in which case the original document will be modified)
Returns
false on success, true if an error occurred
Example of partial update

Take the JSON document { "a": "x", "b": "y", "c": "z" }, whose serialized representation looks like the following:

        0x00 - type: JSONB_TYPE_SMALL_OBJECT
        0x03 - number of elements (low byte)
        0x00 - number of elements (high byte)
        0x22 - number of bytes (low byte)
        0x00 - number of bytes (high byte)
        0x19 - offset of key "a" (high byte)
        0x00 - offset of key "a" (low byte)
        0x01 - length of key "a" (high byte)
        0x00 - length of key "a" (low byte)
        0x1a - offset of key "b" (high byte)
        0x00 - offset of key "b" (low byte)
        0x01 - length of key "b" (high byte)
        0x00 - length of key "b" (low byte)
        0x1b - offset of key "c" (high byte)
        0x00 - offset of key "c" (low byte)
        0x01 - length of key "c" (high byte)
        0x00 - length of key "c" (low byte)
        0x0c - type of value "a": JSONB_TYPE_STRING
        0x1c - offset of value "a" (high byte)
        0x00 - offset of value "a" (low byte)
        0x0c - type of value "b": JSONB_TYPE_STRING
        0x1e - offset of value "b" (high byte)
        0x00 - offset of value "b" (low byte)
        0x0c - type of value "c": JSONB_TYPE_STRING
        0x20 - offset of value "c" (high byte)
        0x00 - offset of value "c" (low byte)
        0x61 - first key  ('a')
        0x62 - second key ('b')
        0x63 - third key  ('c')
        0x01 - length of value "a"
        0x78 - contents of value "a" ('x')
        0x01 - length of value "b"
        0x79 - contents of value "b" ('y')
        0x01 - length of value "c"
        0x7a - contents of value "c" ('z')

We remove the member with name 'b' from the document, using a statement such as:

UPDATE t SET j = JSON_REMOVE(j, '$.b')

This function will then remove the element by moving the key entries and value entries that follow the removed member so that they overwrite the existing entries, and the element count is decremented.

The resulting binary document will look like this:

        0x00 - type: JSONB_TYPE_SMALL_OBJECT
CHANGED 0x02 - number of elements (low byte)
        0x00 - number of elements (high byte)
        0x22 - number of bytes (low byte)
        0x00 - number of bytes (high byte)
        0x19 - offset of key "a" (high byte)
        0x00 - offset of key "a" (low byte)
        0x01 - length of key "a" (high byte)
        0x00 - length of key "a" (low byte)
CHANGED 0x1b - offset of key "c" (high byte)
CHANGED 0x00 - offset of key "c" (low byte)
CHANGED 0x01 - length of key "c" (high byte)
CHANGED 0x00 - length of key "c" (low byte)
CHANGED 0x0c - type of value "a": JSONB_TYPE_STRING
CHANGED 0x1c - offset of value "a" (high byte)
CHANGED 0x00 - offset of value "a" (low byte)
CHANGED 0x0c - type of value "c": JSONB_TYPE_STRING
CHANGED 0x20 - offset of value "c" (high byte)
CHANGED 0x00 - offset of value "c" (low byte)
(free)  0x00
(free)  0x0c
(free)  0x1e
(free)  0x00
(free)  0x0c
(free)  0x20
(free)  0x00
        0x61 - first key  ('a')
(free)  0x62
        0x63 - third key  ('c')
        0x01 - length of value "a"
        0x78 - contents of value "a" ('x')
(free)  0x01
(free)  0x79
        0x01 - length of value "c"
        0x7a - contents of value "c" ('z')

Two binary diffs will be created. One diff changes the element count, and one diff changes the key and value entries.

◆ type()

enum_type json_binary::Value::type ( ) const
inline

◆ update_in_shadow()

bool json_binary::Value::update_in_shadow ( const Field_json field,
size_t  pos,
Json_wrapper new_value,
size_t  data_offset,
size_t  data_length,
const char *  original,
char *  destination,
bool changed 
) const

Update a value in an array or object.

The updated value is written to a shadow copy. The original array or object is left unchanged, unless the shadow copy is actually a pointer to the array backing this Value object. It is assumed that the shadow copy is at least as big as the original document, and that there is enough space at the given position to hold the new value.

Typically, if a document is modified multiple times in a single update statement, the first invocation of update_in_shadow() will have a Value object that points into the binary data in the Field, and write to a separate destination buffer. Subsequent updates of the document will have a Value object that points to the partially updated value in the destination buffer, and write the new modifications to the same buffer.

All changes made to the binary value are recorded as binary diffs using TABLE::add_binary_diff().

Parameters
fieldthe column that is updated
posthe element to update
new_valuethe new value of the element
data_offsetwhere to write the value (offset relative to the beginning of the array or object, obtained with has_space) or zero if the value can be inlined
data_lengththe length of the new value in bytes or zero if the value can be inlined
originalpointer to the start of the JSON document
destinationpointer to the shadow copy of the JSON document (it could be the same as original, in which case the original document will be modified)
[out]changedgets set to true if a change was made to the document, or to false if this operation was a no-op
Returns
false on success, true if an error occurred
Example of partial update

Given the JSON document [ "abc", "def" ], which is serialized like this in a JSON column:

0x02 - type: small JSON array
0x02 - number of elements (low byte)
0x00 - number of elements (high byte)
0x12 - number of bytes (low byte)
0x00 - number of bytes (high byte)
0x0C - type of element 0 (string)
0x0A - offset of element 0 (low byte)
0x00 - offset of element 0 (high byte)
0x0C - type of element 1 (string)
0x0E - offset of element 1 (low byte)
0x00 - offset of element 1 (high byte)
0x03 - length of element 0
'a'
'b'  - content of element 0
'c'
0x03 - length of element 1
'd'
'e'  - content of element 1
'f'

Let's change element 0 from "abc" to "XY" using the following statement:

UPDATE t SET j = JSON_SET(j, '$[0]', 'XY')

Since we're replacing one string with a shorter one, we can just overwrite the length byte with the new length, and the beginning of the original string data. Since the original string "abc" is longer than the new string "XY", we'll have a free byte at the end of the string. This byte is left as is ('c'). The resulting binary representation looks like this:

        0x02 - type: small JSON array
        0x02 - number of elements (low byte)
        0x00 - number of elements (high byte)
        0x12 - number of bytes (low byte)
        0x00 - number of bytes (high byte)
        0x0C - type of element 0 (string)
        0x0A - offset of element 0 (low byte)
        0x00 - offset of element 0 (high byte)
        0x0C - type of element 1 (string)
        0x0E - offset of element 1 (low byte)
        0x00 - offset of element 1 (high byte)
CHANGED 0x02 - length of element 0
CHANGED 'X'
CHANGED 'Y'  - content of element 0
(free)  'c'
        0x03 - length of element 1
        'd'
        'e'  - content of element 1
        'f'

This change will be represented as one binary diff that covers the three changed bytes.

Let's now change element 1 from "def" to "XYZW":

UPDATE t SET j = JSON_SET(j, '$[1]', 'XYZW')

Since the new string is one byte longer than the original string, we cannot simply overwrite the old one. But we can reuse the free byte from the previous update, which is immediately preceding the original value.

To make use of this, we need to change the offset of element 1 to point to the free byte. Then we can overwrite the free byte and the original string data with the new length and string contents. Resulting binary representation:

        0x02 - type: small JSON array
        0x02 - number of elements (low byte)
        0x00 - number of elements (high byte)
        0x12 - number of bytes (low byte)
        0x00 - number of bytes (high byte)
        0x0C - type of element 0 (string)
        0x0A - offset of element 0 (low byte)
        0x00 - offset of element 0 (high byte)
        0x0C - type of element 1 (string)
CHANGED 0x0D - offset of element 1 (low byte)
        0x00 - offset of element 1 (high byte)
        0x02 - length of element 0
        'X'  - content of element 0
        'Y'  - content of element 0
CHANGED 0x04 - length of element 1
CHANGED 'X'
CHANGED 'Y'
CHANGED 'Z'  - content of element 1
CHANGED 'W'

This change will be represented as two binary diffs. One diff for changing the offset, and one for changing the contents of the string.

Then let's replace the string in element 1 with a small number:

UPDATE t SET j = JSON_SET(j, '$[1]', 456)

This will change the type of element 1 from string to int16. Such small numbers are inlined in the value entry, where we normally store the offset of the value. The offset section of the value entry is therefore changed to hold the number 456. The length and contents of the original value ("XYZW") are not touched, but they are now unused and free to be reused. Resulting binary representation:

        0x02 - type: small JSON array
        0x02 - number of elements (low byte)
        0x00 - number of elements (high byte)
        0x12 - number of bytes (low byte)
        0x00 - number of bytes (high byte)
        0x0C - type of element 0 (string)
        0x0A - offset of element 0 (low byte)
        0x00 - offset of element 0 (high byte)
CHANGED 0x05 - type of element 1 (int16)
CHANGED 0xC8 - value of element 1 (low byte)
CHANGED 0x01 - value of element 1 (high byte)
        0x02 - length of element 0
        'X'  - content of element 0
        'Y'  - content of element 0
(free)  0x04 - length of element 1
(free)  'X'
(free)  'Y'
(free)  'Z'  - content of element 1
(free)  'W'

The change is represented as one binary diff that changes the value entry (type and inlined value).

◆ value_entry_offset()

size_t json_binary::Value::value_entry_offset ( size_t  pos) const
inlineprivate

Get the offset of the value entry that describes the element at a given position in this array or object.

Parameters
posthe position of the element
Returns
the offset of the entry, relative to the start of the array or object

Member Data Documentation

◆ @38

union { ... }

◆ m_data

const char* json_binary::Value::m_data

Pointer to the start of the binary representation of the value.

Only used by STRING, OPAQUE, OBJECT and ARRAY.

The memory pointed to by this member is not owned by this Value object. Callers that create Value objects must make sure that the memory is not freed as long as the Value object is alive.

◆ m_double_value

double json_binary::Value::m_double_value

The value if the type is DOUBLE.

◆ m_element_count

uint32 json_binary::Value::m_element_count
private

Element count for arrays and objects.

Unused for other types.

◆ m_field_type

enum_field_types json_binary::Value::m_field_type
private

The MySQL field type of the value, in case the type of the value is OPAQUE.

Otherwise, it is unused.

◆ m_int_value

int64 json_binary::Value::m_int_value

The value if the type is INT or UINT.

◆ m_large

bool json_binary::Value::m_large
private

True if an array or an object uses the large storage format with 4 byte offsets instead of 2 byte offsets.

◆ m_length

uint32 json_binary::Value::m_length
private

The full length (in bytes) of the binary representation of an array or object, or the length of a string or opaque value.

Unused for other types.

◆ m_type

enum_type json_binary::Value::m_type
private

The JSON type of the value.


The documentation for this class was generated from the following files: