HeatWave User Guide  /  ...  /  MySQL HeatWave Lakehouse Error Messages

11.1.5 MySQL HeatWave Lakehouse Error Messages

See MySQL 9.5 Error Message Reference for Lakehouse error messages.

As of MySQL 9.2.2, a detailed summary of warnings is available when loading data to Lakehouse. This includes warnings related to the schema inference stage of loading data.

The summaries include the following:

  • Warnings for Lakehouse Load and Lakehouse Schema Inference.

  • The number of warnings.

  • The code of each warning with a description.

  • If warning messages were filtered, the number of warnings that were not recorded due to the filters.

Refer to the following samples of warning summaries.

A warning summary for Lakehouse Load that includes the number of filtered warnings that are not recorded for the summary. 

Warning 6434    [WARNINGS SUMMARY] Lakehouse Load had 27 warning(s) out of which 27 were not recorded (due to max_error_count limit or filtering rules)
Warning 6434    [WARNINGS SUMMARY] 22 warnings with code: 6053(ER_LH_WARN_EXCEEDS_MAX_TRUNCATING)
Warning 6434    [WARNINGS SUMMARY] 5 warnings with code: 6052(ER_LH_WARN_EXCEEDS_MIN_TRUNCATING)

A warning summary for Lakehouse Schema Inference with Autopilot that includes the number of filtered warnings that are not recorded for the summary. 

{"message": "[WARNINGS SUMMARY] Lakehouse Schema Inference had 18 warning(s) out of which 1 were not recorded (due to max_error_count limit or filtering rules)", "table_name": "lineitem", "schema_name": "autoschema_tmp_db", "condition_no": 1}
{"message": "[WARNINGS SUMMARY] 16 warnings with code: 6062(ER_LH_COLUMN_MISMATCH_ERR)", "table_name": "lineitem", "schema_name": "autoschema_tmp_db", "condition_no": 2}
{"message": "[WARNINGS SUMMARY] 1 warnings with code: 6095(ER_LH_WARN_INFER_SKIPPED_LINES)", "table_name": "lineitem", "schema_name": "autoschema_tmp_db", "condition_no": 3}
{"message": "[WARNINGS SUMMARY] 1 warnings with code: 6063(ER_LH_COLUMN_MAX_ERR)", "table_name": "lineitem", "schema_name": "autoschema_tmp_db", "condition_no": 4}

As of MySQL 9.1.2, the following details are included in Lakehouse error and warning messages related to data files in the Object Storage bucket to help you understand them better:

  • Byte offset text message is added to the existing error/warning messages in csv or JSON file formats.

  • Error/warning position details are added to JSON data type related errors/warnings in csv, JSON, parquet or AVRO file formats.

  • Row number and character position details are added to warning messages(Non strict mode) in csv and JSON file formats.

  • Row number is added to warning messages(Non strict mode) in parquet or AVRO file formats.

Refer to the following sample error messages, for more information.

  • Error message - Byte offset (17) is added to csv or JSON error message. 

    ERROR HY000: Column 1 of {object: "mtrtest/datalake_integer.test_3.tbl" in 
namespace: "OCI_OBJECT_STORE_SOURCE_NAMESPACE", bucket: 
"OCI_OBJECT_STORE_SOURCE_BUCKET"} at byte offset 17 : Invalid integer value.

  • Warning message - Row number (13), character position (1), and byte offset (66) are added to csv or JSON warning message. 

    Warning	6057	Row 13, char 1, column 1 of {object: "mtrtest/datalake_int_basic/0.tbl" in 
namespace: "OCI_OBJECT_STORE_SOURCE_NAMESPACE", bucket: 
"OCI_OBJECT_STORE_SOURCE_BUCKET"} at byte offset 66 : Integer value was truncated.

  • Warning message - Position (11), character (12), are added to csv, JSON, parquet or AVRO warning message. 

    Warning	6121	Row 20, column 3 of {object: 
"datalake_avro_json/datalake_json_error.avro.tbl" in namespace: 
"OCI_OBJECT_STORE_SOURCE_NAMESPACE", bucket: "OCI_OBJECT_STORE_SOURCE_BUCKET"} : 
Invalid escape character in string at position 11(char 12) in this column.

Some Lakehouse error and warning messages include the URL to the external object. The URL can contain up to 1024 characters, and exceed the error message limit of 512 characters. As of MySQL 9.0.1-u1, the truncated message will include a MySQL command to access the full error message. For example:

  • Error message: 

    ERROR HY000: Message was truncated, use 
<SELECT data FROM performance_schema.error_log 
WHERE thread_id=15 AND data LIKE 'ERR_LH_test.t1_long_name_6045%' 
ORDER BY LOGGED DESC>
to obtain more detailed information about the warning/error

  • Warning message: 

    Warning	6045	Message was truncated, use 
<SELECT data FROM performance_schema.error_log 
WHERE thread_id=15 AND data LIKE 'WARN_LH_test.t1_long_name_6045_0%' 
ORDER BY LOGGED DESC>
to obtain more detailed information about the warning/error

As of MySQL 9.0.1-u1, it is possible to filter out Lakehouse warning messages from the console, MySQL Shell and error log following a load command. The console Total Warnings will include both the displayed and the filtered warnings count. This does not filter error messages.

Use the lakehouse_filter_warning_codes_list session variable to filter warning messages by error code. For example: 

mysql> SET @@session.lakehouse_filter_warning_codes_list = '6017,5099,6018';

Use the lakehouse_filter_warning_modes_list session variable to filter warning messages by mode. For example: 

mysql> SET @@session.lakehouse_filter_warning_modes_list = 
       'NUMERIC_TRUNCATION,STRING_TRUNCATION,EMPTY_FILES,DUPLICATE_FILES';

The supported modes are:

  • Mode: AVRO_FILE_BLOCK_HEADER filters the following warning messages:

    • ER_LH_AVRO_CANNOT_PARSE_HEADER

    • ER_LH_AVRO_HEADER_METADATA_ERR

    • ER_LH_FORMAT_HEADER_NO_MAGIC_BYTES

    • ER_LH_AVRO_HEADER_NO_SCHEMA

    • ER_LH_AVRO_INVALID_BLOCK_RECORD_COUNT

    • ER_LH_AVRO_INVALID_BLOCK_SIZE

    • ER_LH_AVRO_NO_CODEC_IN_HEADER

  • Mode: DUPLICATE_FILES filters the following warning messages:

    • ER_LH_DUPLICATE_FILE

  • Mode: EMPTY_FILES filters the following warning messages:

    • ER_LH_EMPTY_FILE

  • Mode: EMPTY_MISSING_COLUMNS filters the following warning messages:

    • ER_LH_COLUMN_MISMATCH_ERR

    • ER_LH_COLUMN_MAX_ERR

    • ER_LH_WARN_COL_MISSING_NOT_NULLABLE

    • ER_LH_COL_IS_EMPTY_WARN

  • Mode: INFER_SKIPPED filters the following warning messages:

    • ER_LH_INFER_FILE_HAS_NO_DATA

    • ER_LH_WARN_INFER_SKIPPED_FILES

    • ER_LH_WARN_INFER_SKIPPED_LINES

  • Mode: MISSING_FILES filters the following warning messages:

    • ER_LH_NO_FILES_FOUND

  • Mode: NULL_COLUMNS filters the following warning messages:

    • ER_LH_COL_NOT_NULLABLE

  • Mode: NUMERIC_PARSING for the int, decimal, float, and double data types filters the following warning messages:

    • ER_LH_BAD_VALUE

    • ER_LH_DECIMAL_OOM_ERR

    • ER_LH_DECIMAL_UNKNOWN_ERR

    • ER_LH_PARQUET_DECIMAL_CONVERSION_ERR

    • ER_LH_REAL_IS_NAN

  • Mode: NUMERIC_TRUNCATION for the int, decimal, float, and double data types filters the following warning messages:

    • ER_LH_DECIMAL_PRECISION_EXCEEDS_SCHEMA

    • ER_LH_EXCEEDS_MAX

    • ER_LH_EXCEEDS_MIN

    • ER_LH_OUT_OF_RANGE

    • ER_LH_WARN_DECIMAL_ROUNDING

    • ER_LH_WARN_EXCEEDS_MIN_TRUNCATING

    • ER_LH_WARN_EXCEEDS_MAX_TRUNCATING

    • ER_LH_WARN_TRUNCATED

  • Mode: STRING_PARSING for the char, enum, text, and varchar data types filters the following warning messages:

    • ER_LH_BAD_VALUE

    • ER_LH_CANNOT_CONVERT_STRING

  • Mode: STRING_TRUNCATION for the char, text, and varchar data types filters the following warning messages:

    • ER_LH_STRING_TOO_LONG

  • Mode: TEMPORAL_PARSING for the date, datetime, time, and timestamp data types filters the following warning messages:

    • ER_LH_BAD_VALUE

    • ER_LH_DATETIME_FORMAT

  • Mode: TEMPORAL_TRUNCATION for the date, datetime, time, and timestamp data types filters the following warning messages:

    • ER_LH_WARN_TRUNCATED

  • Mode: PARQUET_SCHEMA filters the following warning messages:

    • ER_LH_PARQUET_CANNOT_LOCATE_SCHEMA

    • ER_LH_PARQUET_SCHEMA_MISMATCH


PREV   HOME   UP   NEXT
Related Documentation
HeatWave Release Notes
Download this Manual
PDF (US Ltr) - 3.9Mb
PDF (A4) - 3.9Mb