MySQL :: HeatWave User Guide :: 4.7.1 Running HeatWave Chat

HeatWave User Guide / ... / Running HeatWave Chat

4.7.1 Running HeatWave Chat

When you run HeatWave Chat, it automatically loads the mistral-7b-instruct-v1 LLM.

By default, HeatWave Chat searches for an answer to a query across all ingested documents by automatically discovering available vector stores, and returns the answer along with relevant citations. You can limit the scope of search to specific document collections available in certain vector stores or specify documents to include in the search.

As of MySQL 9.2.1, HeatWave Chat lets you use your own embedding tables for context retrieval. And, it uses only the name of the embedding model used to embed the input query to find relevant tables.

In previous versions of MySQL, HeatWave Chat uses only inbuilt vector store tables for context retrieval. And, it uses both the language specified for generating the output and the name of the embedding model used to embed the input query to find relevant tables containing information in the same language and vector embeddings from the sameembedding model.

If you do not have a vector store or an embedding table set up, then HeatWave Chat uses information available in public data sources to generate a response for your query.

Before You Begin

Review the Requirements.
If you want to extend the vector search functionality and ask specific questions about the information available in your proprietary documents that are stored in the vector store, complete the steps to set up a vector store.
To use your own embedding table for context retrieval, create a table that satisfies the following criteria:
- To qualify as a valid embedding table, the table must contain the following columns:
  - A string column containing the text segments.
  - A vector column containing the vector embeddings of the text segments.
- A comment on the vector column must specify the name of the embedding model used to generate the vector embeddings.
- The vector embeddings in your embedding table must be from an embedding model supported by HeatWave. To view the list of available embedding models, see HeatWave In-Database Embedding Models and OCI Generative AI Service Embedding Models.
Following is an example of a valid embedding table that can be used for context retrieval:
```
CREATE TABLE demo_table (id INT AUTO_INCREMENT, demo_text TEXT, primary key (id));
INSERT INTO demo_table (demo_text) VALUES('What is MySQL?');
INSERT INTO demo_table (demo_text) VALUES('What is HeatWave?');
INSERT INTO demo_table (demo_text) VALUES('What is HeatWave GenAI?');
call sys.ML_EMBED_TABLE('demo_schema.demo_table.demo_text', 'demo_schema.demo_table.demo_embedding', JSON_OBJECT('model_id', 'all_minilm_l12_v2'));
```
To learn how to generate vector embeddings and embedding tables using HeatWave GenAI, see Generating Vector Embeddings.

If you want to use both inbuilt vector store tables and your own embedding tables for context retrieval, your embedding table must satisfy the following additional requirements:
- Since the inbuilt vector store tables set up using Asynchronous Load or Auto Parallel Load use predefined column names, the column names in your embedding tables must match the predefined inbuilt vector store table column names as given below:
  - segment: name of the mandatory string column containing the text segments.
  - segment_embedding: name of the mandatory vector column containing the vector embeddings of the text segments.
  - document_name: name of the optional column containing the document names. This column can be of any data type supported by HeatWave.
  - document_id: name of the optional integer column containing the document IDs.
  - metadata: name of the optional JSON column containing metadata for the table.
  - segment_number: name of the optional integer column containing segment number.
- The vector embeddings in your embedding table must be from the same embedding model as the vector store table.

Running the Chat

To run HeatWave Chat, perform the following steps:

To delete previous chat output and state, if any, reset the @chat_options variable:
```
set @chat_options=NULL;
```
Note

Ensure that you use the name @chat_options for the variable. The HEATWAVE_CHAT routine reserves this variable for specifying and saving various chat parameter settings.
Set the @chat_options variable in the following scenarios:
- To use a language other than English, set the language model option:
```
set @chat_options = JSON_OBJECT("model_options", JSON_OBJECT("language", "Language"));
```
  Replace Language with the two-letter ISO 639-1 code for the language you want to use. Default language is en, which is English. To view the list of supported languages, see Languages.
  
  For example, to use French set language to fr:
  
  The language parameter is available in MySQL 9.0.1-u1 and later versions.
```
set @chat_options = JSON_OBJECT("model_options", JSON_OBJECT("language", "fr"));
```
  This resets the @chat_options variable, and specifies the language for the chat.
- To change the column names used by the HEATWAVE_CHAT routine to filter tables for context retrieval.
  
  By default, the routine uses all the predefined vector store column names to filter tables for context retrieval.
  
  To specify the column names used for filtering tables, you can set the vector_store_columns parameter:
```
set @chat_options = JSON_OBJECT("vector_store_columns", JSON_OBJECT("segment", "TextSegmentColumnName", "segment_embedding", "VectorEmbeddingColumnName"), "embed_model_id", "EmbeddingModelName");
```
  Replace the following:
  - TextSegmentColumnName: the name of the embedding table column that contains the text segments in natural language. If multiple tables contain a string column with the same name, they are all used for context retrieval. Default value is segment.
  - VectorEmbeddingColumnName: the name of the embedding table column that contains vector embeddings of the natural-language text segments. If multiple tables contain a vector column with the same name which contain embeddings from the specified embedding model, they are all used for context retrieval. Default value is segment_embedding.
  - EmbeddingModelName: the name of the embedding model to use to generate the vector embeddings for the input query. The routine uses this embedding model name to find tables generated using the same model for context retrieval. By default, the routine uses minilm if the output language is set to English and multilingual-e5-small if the output language is set to a language other than English.
  For example:
```
set @chat_options = JSON_OBJECT("vector_store_columns", JSON_OBJECT("segment", "demo_text", "segment_embedding", "demo_embeddings"), "embed_model_id", "all_minilm_l12_v2");
```
  This resets the @chat_options variable to specify the column names used for filtering tables for context retrieval. In this example, all embedding tables containing a string column demo_text and a vector column demo_embeddings which contains vector embeddings from all_minilm_l12_v2 are used for context retrieval.
  
  However, since the inbuilt vector store tables use predefined column names, if you change a column name used for filtering tables to any value other than the default value, the inbuilt vector store tables are filtered out and are not used for context retrieval.

Then, add your query to HeatWave Chat by using the HEATWAVE_CHAT routine:

call sys.HEATWAVE_CHAT("YourQuery");

For example:

call sys.HEATWAVE_CHAT("What is HeatWave AutoML?");

The output looks similar to the following:

|  HeatWave AutoML is a feature of MySQL HeatWave that makes it easy
to use machine learning, whether you are a novice user or an
experienced ML practitioner. It analyzes the characteristics of the
data and creates an optimized machine learning model that can be used
to generate predictions and explanations. The data and models never
leave MySQL HeatWave, saving time and effort while keeping the data
and models secure. HeatWave AutoML is optimized for HeatWave shapes
and scaling, and all processing is performed on the HeatWave Cluster. |

Repeat this step to ask follow-up questions using the HEATWAVE_CHAT routine:

call sys.HEATWAVE_CHAT("What learning algorithms does it use?");

The output looks similar to the following:

|  HeatWave AutoML uses a variety of machine learning algorithms. It
leverages Oracle AutoML technology which includes a range of algorithms
such as decision trees, random forests, neural networks, and support vector
machines (SVMs). The specific algorithm used by HeatWave AutoML depends on
the characteristics of the data being analyzed and the goals of the model
being created. |

PREV HOME UP NEXT