MySQL :: HeatWave User Guide :: 5.9 HeatWave Lakehouse Error Messages

HeatWave User Guide / HeatWave Lakehouse / HeatWave Lakehouse Error Messages

5.9 HeatWave Lakehouse Error Messages

See MySQL 9.0 Error Message Reference for Lakehouse error messages.

As of MySQL 9.2.2, a detailed summary of warnings is available when loading data to Lakehouse. This includes warnings related to the schema inference stage of loading data.

The summaries include the following:

Warnings for Lakehouse Load and Lakehouse Schema Inference.
The number of warnings.
The code of each warning with a description.
If warning messages were filtered, the number of warnings that were not recorded due to the filters.

Refer to the following samples of warning summaries.

A warning summary for Lakehouse Load that includes the number of filtered warnings that are not recorded for the summary.

Press CTRL+C to copy
 
Warning 6434    [WARNINGS SUMMARY] Lakehouse Load had 27 warning(s) out of which 27 were not recorded (due to max_error_count limit or filtering rules)
Warning 6434    [WARNINGS SUMMARY] 22 warnings with code: 6053(ER_LH_WARN_EXCEEDS_MAX_TRUNCATING)
Warning 6434    [WARNINGS SUMMARY] 5 warnings with code: 6052(ER_LH_WARN_EXCEEDS_MIN_TRUNCATING)

A warning summary for Lakehouse Schema Inference with Autopilot that includes the number of filtered warnings that are not recorded for the summary.

Press CTRL+C to copy
 
{"message": "[WARNINGS SUMMARY] Lakehouse Schema Inference had 18 warning(s) out of which 1 were not recorded (due to max_error_count limit or filtering rules)", "table_name": "lineitem", "schema_name": "autoschema_tmp_db", "condition_no": 1}
{"message": "[WARNINGS SUMMARY] 16 warnings with code: 6062(ER_LH_COLUMN_MISMATCH_ERR)", "table_name": "lineitem", "schema_name": "autoschema_tmp_db", "condition_no": 2}
{"message": "[WARNINGS SUMMARY] 1 warnings with code: 6095(ER_LH_WARN_INFER_SKIPPED_LINES)", "table_name": "lineitem", "schema_name": "autoschema_tmp_db", "condition_no": 3}
{"message": "[WARNINGS SUMMARY] 1 warnings with code: 6063(ER_LH_COLUMN_MAX_ERR)", "table_name": "lineitem", "schema_name": "autoschema_tmp_db", "condition_no": 4}

As of MySQL 9.1.2, the following details are included in Lakehouse error and warning messages related to data files in the Object Storage bucket to help you understand them better:

Byte offset text message is added to the existing error/warning messages in csv or JSON file formats.
Error/warning position details are added to JSON data type related errors/warnings in csv, JSON, parquet or AVRO file formats.
Row number and character position details are added to warning messages(Non strict mode) in csv and JSON file formats.
Row number is added to warning messages(Non strict mode) in parquet or AVRO file formats.

Refer to the following sample error messages, for more information.

Error message - Byte offset (17) is added to csv or JSON error message.

Press CTRL+C to copy
 
ERROR HY000: Column 1 of {object: "mtrtest/datalake_integer.test_3.tbl" in 
namespace: "OCI_OBJECT_STORE_SOURCE_NAMESPACE", bucket: 
"OCI_OBJECT_STORE_SOURCE_BUCKET"} at byte offset 17 : Invalid integer value.

Warning message - Row number (13), character position (1), and byte offset (66) are added to csv or JSON warning message.

Press CTRL+C to copy
 
Warning	6057	Row 13, char 1, column 1 of {object: "mtrtest/datalake_int_basic/0.tbl" in 
namespace: "OCI_OBJECT_STORE_SOURCE_NAMESPACE", bucket: 
"OCI_OBJECT_STORE_SOURCE_BUCKET"} at byte offset 66 : Integer value was truncated.

Warning message - Position (11), character (12), are added to csv, JSON, parquet or AVRO warning message.

Press CTRL+C to copy
 
Warning	6121	Row 20, column 3 of {object: 
"datalake_avro_json/datalake_json_error.avro.tbl" in namespace: 
"OCI_OBJECT_STORE_SOURCE_NAMESPACE", bucket: "OCI_OBJECT_STORE_SOURCE_BUCKET"} : 
Invalid escape character in string at position 11(char 12) in this column.

Some Lakehouse error and warning messages include the URL to the external object. The URL can contain up to 1024 characters, and exceed the error message limit of 512 characters. As of MySQL 9.0.1-u1, the truncated message will include a MySQL command to access the full error message. For example:

Error message:

Press CTRL+C to copy
 
ERROR HY000: Message was truncated, use 
<SELECT data FROM performance_schema.error_log 
WHERE thread_id=15 AND data LIKE 'ERR_LH_test.t1_long_name_6045%' 
ORDER BY LOGGED DESC>
to obtain more detailed information about the warning/error

Warning message:

Press CTRL+C to copy
 
Warning	6045	Message was truncated, use 
<SELECT data FROM performance_schema.error_log 
WHERE thread_id=15 AND data LIKE 'WARN_LH_test.t1_long_name_6045_0%' 
ORDER BY LOGGED DESC>
to obtain more detailed information about the warning/error

As of MySQL 9.0.1-u1, it is possible to filter out Lakehouse warning messages from the console, MySQL Shell and error log following a load command. The console Total Warnings will include both the displayed and the filtered warnings count. This does not filter error messages.

Use the lakehouse_filter_warning_codes_list session variable to filter warning messages by error code. For example:

Press CTRL+C to copy
 
mysql> SET @@session.lakehouse_filter_warning_codes_list = '6017,5099,6018';

Use the lakehouse_filter_warning_modes_list session variable to filter warning messages by mode. For example:

Press CTRL+C to copy
 
mysql> SET @@session.lakehouse_filter_warning_modes_list = 
       'NUMERIC_TRUNCATION,STRING_TRUNCATION,EMPTY_FILES,DUPLICATE_FILES';

The supported modes are:

Mode: AVRO_FILE_BLOCK_HEADER filters the following warning messages:
- ER_LH_AVRO_CANNOT_PARSE_HEADER
- ER_LH_AVRO_HEADER_METADATA_ERR
- ER_LH_FORMAT_HEADER_NO_MAGIC_BYTES
- ER_LH_AVRO_HEADER_NO_SCHEMA
- ER_LH_AVRO_INVALID_BLOCK_RECORD_COUNT
- ER_LH_AVRO_INVALID_BLOCK_SIZE
- ER_LH_AVRO_NO_CODEC_IN_HEADER
Mode: DUPLICATE_FILES filters the following warning messages:
- ER_LH_DUPLICATE_FILE
Mode: EMPTY_FILES filters the following warning messages:
- ER_LH_EMPTY_FILE
Mode: EMPTY_MISSING_COLUMNS filters the following warning messages:
- ER_LH_COLUMN_MISMATCH_ERR
- ER_LH_COLUMN_MAX_ERR
- ER_LH_WARN_COL_MISSING_NOT_NULLABLE
- ER_LH_COL_IS_EMPTY_WARN
Mode: INFER_SKIPPED filters the following warning messages:
- ER_LH_INFER_FILE_HAS_NO_DATA
- ER_LH_WARN_INFER_SKIPPED_FILES
- ER_LH_WARN_INFER_SKIPPED_LINES
Mode: MISSING_FILES filters the following warning messages:
- ER_LH_NO_FILES_FOUND
Mode: NULL_COLUMNS filters the following warning messages:
- ER_LH_COL_NOT_NULLABLE
Mode: NUMERIC_PARSING for the int, decimal, float, and double data types filters the following warning messages:
- ER_LH_BAD_VALUE
- ER_LH_DECIMAL_OOM_ERR
- ER_LH_DECIMAL_UNKNOWN_ERR
- ER_LH_PARQUET_DECIMAL_CONVERSION_ERR
- ER_LH_REAL_IS_NAN
Mode: NUMERIC_TRUNCATION for the int, decimal, float, and double data types filters the following warning messages:
- ER_LH_DECIMAL_PRECISION_EXCEEDS_SCHEMA
- ER_LH_EXCEEDS_MAX
- ER_LH_EXCEEDS_MIN
- ER_LH_OUT_OF_RANGE
- ER_LH_WARN_DECIMAL_ROUNDING
- ER_LH_WARN_EXCEEDS_MIN_TRUNCATING
- ER_LH_WARN_EXCEEDS_MAX_TRUNCATING
- ER_LH_WARN_TRUNCATED
Mode: STRING_PARSING for the char, enum, text, and varchar data types filters the following warning messages:
- ER_LH_BAD_VALUE
- ER_LH_CANNOT_CONVERT_STRING
Mode: STRING_TRUNCATION for the char, text, and varchar data types filters the following warning messages:
- ER_LH_STRING_TOO_LONG
Mode: TEMPORAL_PARSING for the date, datetime, time, and timestamp data types filters the following warning messages:
- ER_LH_BAD_VALUE
- ER_LH_DATETIME_FORMAT
Mode: TEMPORAL_TRUNCATION for the date, datetime, time, and timestamp data types filters the following warning messages:
- ER_LH_WARN_TRUNCATED
Mode: PARQUET_SCHEMA filters the following warning messages:
- ER_LH_PARQUET_CANNOT_LOCATE_SCHEMA
- ER_LH_PARQUET_SCHEMA_MISMATCH

PREV HOME UP NEXT