27.3.8.1 LLM Class

This class represents a large language model. Members accessible from instances of this class are listed here:

• name (String): The name of the model

• options (Object): The model's configuration options

• isLoaded (Boolean): true if the model is loaded, false if it is not

Class constructor:

LLM LLM(name, options)

Arguments

• name: the name of the model

• options (Object) (default {}): an object containing the options used by this instance

Return type

• An instance of LLM

Usage

• let model = MLL("cohere.command", {max_tokens: 10})

• LLM.unload()

• LLM.generate()

• LLM.embed()

• LLM.rag()

LLM.unload()

Unloads the model that was loaded in the constructor. This is optional, but recommended, since doing so can reduce memory usage; after unloading, any subsequent attempt to use the instance raises an error.

Arguments

• None

Return type

• undefined

Usage

• model.unload()

LLM.generate()

This method acts as a wrapper for ML_GENERATE, and generates a response using the prompt and options provided for the loaded model.

Arguments

• prompt (String): prompt to be used for text generation

• options (Object) (default {}): an object containing the options used by this instance; see the description of ML_GENERATE for available options

Return type

• Object: The structure is similar to that of ML_GENERATE.

Usage

• let response = model.generate("What is MySql?", {"top_k": 2, "task": "generation"})

LLM.embed()

This method is a wrapper for ML_EMBED_ROW, and generates an embedding whose type corresponds to the MySQL VECTOR type.

Arguments

• query (String): The text of the query to be embedded

• options (Object) (optional; default {}): an object containing the options used for embedding; see the description of ML_EMBED_ROW for available options

Return type

• Float32Array (MySQL VECTOR): The embedding

Usage

• let embedding = model.embed("What is MySql?")

LLM.rag()

This method performs Retrieval Augmented Generation for a given query using the loaded genAI model, acting as a wrapper of ML_RAG.

Arguments

• query (String): The text of the query to be used for generation

• options (Object) (default {}): The options employed for generation; these follow the same rules as the options used with LLM.generate()

Return type

• Object: The structure is similar to that of the object returned by ML_RAG.

Usage

• let result = model.rag("What is MySql?", {schema: ["vector_store"], n_citations: 1})