2.2 Loading Data to HeatWave MySQL

2.2.1 Prerequisites

2.2.2 Loading Data Manually

2.2.3 Loading Data Using Auto Parallel Load

2.2.4 Monitoring Load Progress

2.2.5 Checking Load Status

2.2.6 Data Compression

2.2.7 Change Propagation

2.2.8 Reload Tables

As of MySQL 9.1.0, Auto Parallel Load uses Autopilot to collect statistics about frequently used InnoDB tables. Auto Parallel Load then automatically loads these tables into HeatWave. If a manually loaded table might cause a memory conflict, Auto Unload unloads automatically loaded tables to free up memory.

HeatWave MySQL provides the following methods to load data to HeatWave:

• Loading data manually. This method loads one table at a time and involves executing multiple statements for each table. See Section 2.2.2, “Loading Data Manually”.

• Loading data using Auto Parallel Load. This HeatWave Autopilot enabled method loads one or more schemas at a time and facilitates loading by automating manual steps and optimizing the number of parallel load threads for a faster load. See Section 2.2.3, “Loading Data Using Auto Parallel Load”.

• For users of HeatWave on AWS, load data using the HeatWave Console. This GUI-based and HeatWave Autopilot enabled method loads selected schemas and tables using an optimized number of parallel load threads for a faster load. See Manage Data in HeatWave with Workspaces in the HeatWave on AWS Service Guide.

HeatWave loads data with batched, multi-threaded reads from InnoDB. HeatWave then converts the data into columnar format and sends it over the network to distribute it among HeatWave nodes in horizontal slices. HeatWave partitions data by the table primary key, unless the table definition includes data placement keys. See Section 2.7.2, “Defining Data Placement Keys”.

Concurrent DML operations and queries on the MySQL node are supported while a data load operation is in progress; however, concurrent operations on the MySQL node can affect load performance and vice versa.

After tables are loaded, changes to table data on the MySQL DB System node are automatically propagated to HeatWave. For more information, see Section 2.2.7, “Change Propagation”.

For each table that is loaded in HeatWave, the default heap segment size is 64KB per table, and this is allocated from the root heap. The root heap available to HeatWave is approximately 400GB.

HeatWave compresses data as it is loaded, which permits HeatWave nodes to store more data at a minor cost to performance. If you do not want to compress data as it is loaded in HeatWave, you must disable compression before loading data. See Section 2.2.6, “Data Compression”.

For related best practices, see Section 2.9, “Best Practices”.