4.8 HeatWave Lakehouse Limitations

The following limitations apply to Lakehouse. For HeatWave limitations, see: Section 2.17, “HeatWave Limitations”.

Lakehouse Limitations for all File Formats

• See Section 2.17.8, “Other Limitations” for all HeatWave query related limitations. Before MySQL 8.3.0, not all queries could be accelerated by HeatWave Lakehouse, and a warning or error message was displayed.

• Do not create Lakehouse tables on the source DB in a replicated DB System if any of the replicas are outside MySQL HeatWave on OCI or MySQL HeatWave on AWS. This will cause replication errors.

• It is not possible to dump external tables using the MySQL Shell export utilities, such as dumpInstance(). External tables are not replicated to InnoDB storage and cannot be exported. To export InnoDB data from a Lakehouse enabled database, exclude the external tables with an excludeTables option.

• It is not possible to restore a backup from a Lakehouse enabled DB System to a standalone DB System.

• Lakehouse enabled DB Systems can support a maximum of 512 nodes.

• HeatWave Lakehouse does not support the following:

  • DML statements:

    • INSERT

    • UPDATE

    • DELETE

    • REPLACE

  • Creating temporary tables.

  • Creating AUTO_INCREMENT columns.

  • Creating triggers.

  • Running ANALYZE TABLE.

  • Running ALTER TABLE statements that construct indexes, ADD or DROP columns, or add enforced check constraints.

  • Lakehouse does not enforce any specified constraints. For example, primary key uniqueness constraints are not enforced. Therefore, GROUP BY queries can have duplicate rows in their results.

  • Running any statement with a STORAGE clause.

  • Lakehouse only supports the MySQL.HeatWave.VM.Standard and HeatWave.512GB shapes.

  • The following limitations only apply to DB Systems up to MySQL 8.3.0-u2. MySQL 8.3.0-u2 removes these limitations:

    • Point-in-time-recovery. All DB Systems are created with point-in-time-recovery enabled by default. To enable Lakehouse, disable point-in-time-recovery.

    • High Availability. To enable Lakehouse, disable High Availability or create a standalone DB System for use with Lakehouse.

    • Read Replication.

    • Outbound Replication.

    Note

    When disabling Read Replicas, High Availability, or Outbound Replication on a DB System to enable Lakehouse, it is recommended to wait several minutes before enabling Lakehouse. This allows replication processes to complete their tasks and for the binlog dump threads to stop.

Lakehouse Limitations for Avro Format Files

• Avro file headers must be less than 1MB.

• The data in an Avro block must be no larger than 64MiB before applying any compression.

• Lakehouse only supports uncompressed Avro blocks, or Avro blocks compressed with Snappy or Deflate algorithms. Lakehouse does not support other compression algorithms.

• Lakehouse does not support these data types in Avro files: Array, Map, and nested records. Lakehouse marks columns with these data types as NOT SECONDARY, and does not load them.

• Lakehouse only supports an Avro union between one supported data type and NULL.

• The Avro DECIMAL data type is limited to the scale and precision supported by MySQL.

• Avro data with FIXED values or values in BYTES are only supported with the Avro DECIMAL data type.

Lakehouse Limitations for CSV Format Files

• MySQL does not recognise NaN values in CSV files. Replace all NaN values with NULL.

• Lakehouse does not support CSV files with more than 4MB per line.

Lakehouse Limitations for Parquet Format Files

• NaN values in Parquet files are loaded as NULL.

• Lakehouse does not support Parquet files with a row group size of more than 500MB.

• Lakehouse does not support these data types in Parquet files: ENUM, UUID, Interval, BSON, List, Map, Unknown. Lakehouse marks columns with these data types as NOT SECONDARY, and does not load them.

• Up to MySQL 8.3.0-u2 Lakehouse does not support the JSON data type in Parquet files. MySQL 8.3.0-u2 removes this limitation.