To keep up with the changes and updates in the documents in your object storage, you must update the vector embeddings loaded in the vector store table on a regular basis. This ensures that the responses generated by HeatWave GenAI are not only accurate, but also up-to-date. And it deletes the embeddings that are no longer useful.
You can update the vector store table using the following methods:
Complete the steps to set up a vector store.
To update the embeddings in the vector store table using incremental load, perform the following steps:
-
Check that the vector embeddings are loaded in the vector store table you want to update:
select count(*) from VectorStoreTableName;
Replace
VectorStoreTableName
with the name of the vector store table you want to update.For example:
select count(*) from demo_embeddings;
If you see a numerical value in the output, the embeddings are loaded in the table.
-
To specify vector store table to update, set the
@dl_tables
session variable:set @dl_tables = '[ { "db_name": "DBName", "tables": [ { "table_name": "VectorStoreTableName" }] } ]';
Replace the following:
DBName
: the name of database that contains the vector store table.VectorStoreTableName
: the vector store table name.
For example:
set @dl_tables = '[ { "db_name": "demo_db", "tables": [ { "table_name": "demo_embeddings" }] } ]';
-
To enable incremental loading, set the
refresh_external_tables
parameter in the@options
session variable:set @options = JSON_OBJECT('mode', 'normal', 'refresh_external_tables', true);
-
To load the new and updated vector embeddings into the vector store, use the
HEATWAVE_LOAD
routine:call sys.HEATWAVE_LOAD(CAST(@dl_tables AS JSON), @options);
This updates the vector store table with the new and updated embeddings.
-
Verify that the embeddings are updated in the vector store table:
select count(*) from VectorStoreTableName;
For example:
select count(*) from demo_embeddings;
If you see a numerical value in the output which is different than the one you saw in step 1, then the vector store table is successfully updated. In some cases, this value might not change even though the vector store table is successfully refreshed.
To delete and recreate the vector store table and vector embeddings, perform the following steps:
-
Delete the vector store table:
drop table VectorStoreTableName;
Replace
VectorStoreTableName
with the vector store table name. To create new embeddings for the updated documents, repeat the steps to set up a vector store.