Related Documentation Download this Manual
PDF (US Ltr) - 2.1Mb
PDF (A4) - 2.1Mb


HeatWave User Guide  /  ...  /  Refreshing Data Using Incremental Load

5.4.1 Refreshing Data Using Incremental Load

After an initial data load, the data can become outdated, and the external tables require a refresh. Subsequent calls to Auto Parallel Load refresh the data. As of MySQL 9.0.0, you can use Lakehouse Incremental Load, to refresh the loaded data.

Lakehouse Incremental Load uses the existing ENGINE_ATTRIBUTE option. See: Section 5.2.2, “Lakehouse External Table Syntax”.

Using Lakehouse Incremental Load, it is possible to add or remove individual files if they still match the defined file pattern or file prefix. However, to learn how to change file parameters, file names, file patterns or file prefixes, use Section 5.4.2, “Adding or Removing Files Using Selective Load” which is supported in MySQL 9.1.2 and later versions.

After the initial data load, a subsequent call to Auto Parallel Load with the refresh_external_tables option set to true refreshes the data, see: Section 2.2.3.2, “Auto Parallel Load Syntax”.

If a subsequent call to Auto Parallel Load includes tables that have not yet been loaded to Lakehouse, then Auto Parallel Load will load them for the first time.

A call to Auto Parallel Load might contain both loaded and unloaded tables. Those that are unloaded will be loaded, and those that are already loaded will be refreshed.