9.3 Refreshing Lakehouse Data

The HeatWave Lakehouse tables contain a snapshot of your data in Amazon S3 at the time you loaded the data into your HeatWave Cluster. If the data has been altered or removed subsequently on Amazon S3, or if you want to add similar data from Amazon S3 that maps to your Lakehouse tables, you can perform a Lakehouse Incremental Load when using MySQL 9.1.0 or later.

In a Lakehouse Incremental Load, it is not possible to change the parameters you used to create the Lakehouse mapping initially (for example, the file type, file path type, or file parsing settings). However, individual files for data sourcing are added or removed according to the original file prefix or file path; for example, if your File path type is Prefix and a new file similar to the original files has been added to the location specified by the Amazon S3 bucket prefix path, the data in the new file will be loaded into the Lakehouse table by the refresh.

Here are the prerequisites for refreshing a HeatWave Lakehouse table:

• What are required for Creating Lakehouse Mapping.
• The privileges to load data into your HeatWave Cluster, as described in Loading or Unloading Data into HeatWave Cluster.
• An existing Lakehouse table in your HeatWave Cluster.
• The DB System that your HeatWave Cluster is associated with must be running MySQL 9.0.0 or later.
• Changes have been made to the files under the Amazon S3 bucket that were used to create Lakehouse mapping initially, or similar files have been added to the same file path.

Do the following to refresh your Lakehouse data with a Lakehouse Incremental Load:

1. Connect to the DB System that is associated with the Lakehouse table to be refreshed. See Connecting from the Console.
2. Click the Data Management tab, and select on it the databases containing the Lakehouse tables you want to refresh. Alternatively, you can expand the table list for each database, and then select specific Lakehouse tables for refresh.
3. Click the Refresh Lakehouse Data button on top. The Refresh Lakehouse Data dialog box shows the following:
   • The DB System which is associated with Lakehouse tables to be refreshed.
   • The number of Tables to refresh.
   • The Schemas and tables to be refreshed.
4. Click Refresh to continue. A message appears when the data refresh is completed.
5. If you want to cancel the refresh before it is completed, click the Actions button on top and then select Cancel Lakehouse Refresh.

Related Topics

• Lakehouse Incremental Load in the HeatWave User Guide

Parent topic: Manage Data in HeatWave