2.2.3 Loading Data Using Auto Parallel Load

2.2.3.1 Auto Parallel Load Requirements

2.2.3.2 Auto Parallel Load Syntax

2.2.3.3 Running Auto Parallel Load

2.2.3.4 Memory Estimation for String Column Encoding

2.2.3.5 Auto Parallel Load Command-Line Help

2.2.3.6 Auto Parallel Load Examples

2.2.3.7 The Auto Parallel Load Report Table

As of MySQL 9.1.0, Auto Parallel Load uses Autopilot to collect statistics about frequently used InnoDB tables. Auto Parallel Load then automatically loads these tables into HeatWave. If a manually loaded table might cause a memory conflict, Auto Unload unloads automatically loaded tables to free up memory.

Auto Parallel Load automatically loads any table that does not have the secondary engine set, or the secondary engine is set to RAPID.

Auto Parallel Load has a 24 hour wait interval between checking the table statistics, and only checks them during a quiet period. Therefore, within 24 hours of the first query that accesses a table, Auto Parallel Load will automatically load the table into HeatWave if there is sufficient memory.

Auto Parallel Load does not automatically load Lakehouse tables.

Auto Parallel Load facilitates the process of loading data into HeatWave by automating many of the steps involved, including:

• Excluding schemas, tables, and columns that cannot be loaded.

• Defining RAPID as the secondary engine for tables that are to be loaded.

• Verifying that there is sufficient memory available for the data.

• Optimizing load parallelism based on machine-learning models.

• Loading data into HeatWave.

Auto Parallel Load, which can be run manually from any MySQL client or connector, is implemented as a stored procedure named heatwave_load, which resides in the MySQL sys schema. Running Auto Parallel Load involves issuing a CALL statement for the stored procedure, which takes schemas and options as arguments; for example, this statement loads the tpch schema:

mysql> CALL sys.heatwave_load(JSON_ARRAY("tpch"), NULL);