5.2.4 Loading Data Using Auto Parallel Load

5.2.4.1 Lakehouse Auto Parallel Load Schema Inference

5.2.4.2 Lakehouse Auto Parallel Load with the external_tables Option

HeatWave Lakehouse extends Auto Parallel Load in two ways:

• Lakehouse Auto Parallel Load includes schema inference which analyzes the external data to infer the table structure.

• Lakehouse Auto Parallel Load uses the external_tables option to enable loading data from external sources. See: Section 5.2.4.2, “Lakehouse Auto Parallel Load with the external_tables Option”. Do not use as of MySQL 8.4.0. external_tables will be deprecated in a future release.

• As of MySQL 8.4.0, Lakehouse Auto Parallel Load uses the db_object with table or exclude_tables instead. See: Section 2.2.3.2, “Auto Parallel Load Syntax”.

Lakehouse Auto Parallel Load facilitates the process of loading data into HeatWave by automating many of the steps involved, including:

• All these steps: Section 2.2.3, “Loading Data Using Auto Parallel Load”.

• Defining lakehouse as the engine for tables that are to be loaded.

• Defining the ENGINE_ATTRIBUTE for tables that are to be loaded.

• Loading CSV files without having to specify the exact record and field delimiters. The detected delimiters are then used in the generated CREATE TABLE statement. If the table is already created, then an ALTER TABLE statement is issued to update the ENGINE_ATTRIBUTE option for the table, which replaces values of delimiter parameters that are set to auto with the detected delimiters. For more information, see Section 5.2.2, “Lakehouse External Table Syntax”. This is supported from MySQL 9.2.0.

Lakehouse Auto Parallel Load includes Lakehouse Incremental Load that can refresh tables after an initial load.