MySQL :: HeatWave User Guide

HeatWave User Guide / ... / ML_RAG

4.7.4 ML_RAG

The ML_RAG routine performs retrieval-augmented generation (RAG) by:

Taking a natural-language query.
Retrieving context from relevant documents using semantic search.
Generating a response that integrates information from the retrieved documents.

This routine provides detailed, accurate, and contextually relevant answers by augmenting a generative model with information retrieved from a comprehensive knowledge base.

ML_RAG Syntax

mysql> call sys.ML_RAG('QueryInNaturalLanguage', 'Output', [options]);

options: {
  JSON_OBJECT('key','value'[,'key','value'] ...)
    'key','value': {
    ['vector_store', JSON_ARRAY('VectorStoreTableName1'[,'VectorStoreTableName2'] ...)]
    ['schema', JSON_ARRAY('Schema1'[,'Schema2'] ...)]
    ['n_citations', NumberOfCitations]
    ['distance_metric', {'COSINE'|'DOT'|'EUCLIDEAN'}]
    ['document_name', JSON_ARRAY('DocumentName1'[,'DocumentName2'] ...)]
    ['skip_generate', {true|false}]
    ['model_options', JSON_OBJECT('Key1','Value1'[,'Key2','Value2'] ...)]
    ['exclude_vector_store', JSON_ARRAY('ExcludeVectorStoreTableName1'[,'ExcludeVectorStoreTableName2'] ...)]
    ['exclude_document_name', JSON_ARRAY('ExcludeDocumentName1'[,'ExcludeDocumentName2'] ...)]
    }
}

Following are ML_RAG parameters:

QueryInNaturalLangugae: specifies the natural-language query.
Output: stores the generated output. The output contains the following segments:
- text: the generated text-based response.
- citations: contains the following details:
  - segment: the textual content that is retrieved from the vector store through semantic search, and used as context generating the response.
  - distance: the distance between the query embedding the segment embedding.
  - document_name: the name of the document from which the segment is retrieved.
- vector_store: the list of vector store tables used for context retrieval.
options: specifies optional parameters as key-value pairs in JSON format. It can include the following parameters:
- vector_store: specifies a list of loaded vector store tables to use for context retrieval. The routine ignores invalid table names. By default, the routine performs a global search across all the available vector store tables in the DB system.
- schema: specifies a list of schemas to check for loaded vector store tables. By default, the routine performs a global search across all the available vector store tables in all the schemas that are available in the DB system.
- n_citations: specifies the number of segments to consider for context retrieval. Default value is 3. Possible values are integer values between 0 and 100.
- distance_metric: specifies the distance metrics to use for context retrieval. Default value is COSINE. Possible values are COSINE, DOT, and EUCLIDEAN.
- document_name: limits the documents to use for context retrieval. Only the specified documents are used. By default, the routine performs a global search across all the available documents stored in all the available vector stores in the DB system.
- skip_generate: specifies whether to skip generation of the text-based response, and only perform context retrieval from the available or specified vector stores, schemas, or documents. Default value is false.
- model_options: additional options that you can set for generating the text-based response. These are the same options that are available in the ML_GENERATE routine, which alter the text-based response per the specified settings. However, the context option is not supported as an ML_RAG model option. Default value is '{"model_id": "mistral-7b-instruct-v1"}'.
- exclude_vector_store: specifies a list of loaded vector store tables to exclude from context retrieval. The routine ignores invalid table names. Default value is NULL. This option is available in MySQL 9.0.1-u1 and later versions.
- exclude_document_name: specifies a list of documents to exclude from context retrieval. Default value is NULL. This option is available in MySQL 9.0.1-u1 and later versions.
- retrieval_options: specifies optional context retrieval parameters as key-value pairs in JSON format. If a parameter value in retrieval_options is set to auto, the default value for that parameter is used.
  
  The retrieval_options parameters are available in MySQL 9.1.2 and later versions.
  
  It can include the following parameters:
  - max_distance: specifies a maximum distance threshold for filtering out segments from context retrieval. Segments for which the distance from the input query exceeds the specified maximum distance threshold are excluded from content retrieval. This ensures that only the segments that are closer to the input query are included during context retrieval. However, if no segments are found within the specified distance, the routine fails to run.
    
    Note
    
    If this parameter is set, the default value of the n_citations parameter is automatically updated to 10.
    
    Default value is 0.6 for all distance metrics.
    
    Possible values are decimal values between 0 and 999999.9999.
  - percentage_distance: specifies what percentage of distance to the nearest segment is to be used to determine the maximum distance threshold for filtering out segments from context retrieval.
    
    Following is the formula used for calculating the maximum distance threshold:
    
    MaximumDistanceThreshold = DistanceOfInputQueryToNearestSegment + [(percentage_distance / 100) * DistanceOfInputQueryToNearestSegment]
    
    Which means that the segments for which the distance to the input query exceeds the distance of the input query to the nearest segment by the specified percentage are filtered out from context retrieval.
    
    Note
    
    If this parameter is set, the default value of the n_citations parameter is automatically updated to 10.
    
    Default value is 20 for all distance metrics.
    
    Possible values are decimal values between 0 and 999999.9999.
    
    Note
    
    If both max_distance and percentage_distance are set, the smaller threshold value is considered for filtering out the segments.
  - segment_overlap: specifies the number of additional segments adjacent to the nearest segments to the input query to be included in context retrieval. These additional segments provide more continuous context for the input query. Default value is 1. Possible values are integer values between 0 and 5.

Syntax Examples

Retrieving context and generating output:

call sys.ML_RAG("What is AutoML",@output,@options);

Where, @options is set to specify the vector store table to use using vector_store key, as shown below:

set @options = JSON_OBJECT("vector_store", JSON_ARRAY("demo_db.demo_embeddings"));

Print the output:

select JSON_PRETTY(@output);

The output of the routine looks similar to the following:

| {
  "text": " AutoML is a machine learning technique that automates the process of selecting, training, and evaluating machine learning models. It involves using algorithms and techniques to automatically identify the best model for a given dataset and optimize its hyperparameters without requiring manual intervention from data analysts or ML practitioners. AutoML can be used in various stages of the machine learning pipeline, including data preprocessing, feature engineering, model selection, hyperparameter tuning, and model evaluation.",
  "citations": [
    {
      "segment": "The output looks similar to the following: \"text\": \" AutoML is a machine learning technique that automates the process of selecting, training, and evaluating machine learning models. It involves using algorithms and techniques to automatically identify the best model for a given dataset and optimize its hyperparameters without requiring manual intervention from data analysts or ML practitioners. AutoML can be used in various stages of the machine learning pipeline, including data preprocessing, feature engineering, model selection,",
      "distance": 0.3082,
      "document_name": "https://objectstorage.Region.oraclecloud.com/n/Namespace/b/BucketName/o/Path/Filename"
    },
    {
      "segment": "hyperparameter tuning, and model evaluation.\", \"citations\": [   { \"segment\": \"Oracle AutoML also produces high quality models very efficiently, which is achieved through a scalable design and intelligent choices that reduce trials at each stage in the pipeline.\\n Scalable design: The Oracle AutoML pipeline is able to exploit both HeatWave internode and intranode parallelism, which improves scalability and reduces runtime.\",     \"distance\": 0.4262576103210449,     \"document_name\":",
      "distance": 0.3927,
      "document_name": "https://objectstorage.Region.oraclecloud.com/n/Namespace/b/BucketName/o/Path/Filename"
    },
    {
      "segment": "110 Chapter 3 HeatWave AutoML\nTable of Contents 3.1 HeatWave AutoML Features ..................................................................................................... 112\n3.1.1 HeatWave AutoML Supervised Learning ......................................................................... 112\n3.1.2 HeatWave AutoML Ease of Use ..................................................................................... 113",
      "distance": 0.3936,
      "document_name": "https://objectstorage.Region.oraclecloud.com/n/Namespace/b/BucketName/o/Path/Filename"
    }
  ],
  "vector_store": [
    "demo_db.demo_embeddings"
  ],
  "retrieval_info": {
    "method": "n_citations",
    "threshold": 0.3936
  }
} |

PREV HOME UP NEXT