scores a model by generating predictions using the feature
columns in a labeled dataset as input and comparing the
predictions to ground truth values in the target column of the
The dataset used with
should have the same feature columns as the dataset used to
train the model but the data sample should be different from
the data used to train the model; for example, you might
reserve 20 to 30 percent of a labeled dataset for scoring.
returns a computed metric indicating the quality of the model.
A value of
None is reported if a score for
the specified or default metric cannot be computed. If an
invalid metric is specified, the following error message is
reported: Invalid data for the metric. Score could
not be computed.
Models with a low score can be expected to perform poorly, producing predictions and explanations that cannot be relied upon. A low score typically indicates that the provided feature columns are not a good predictor of the target values. In this case, consider adding more rows or more informative features to the training dataset.
You can also run
on the training dataset and a labeled test dataset and compare
results to ensure that the test dataset is representative of
the training dataset. A high score on a training dataset and
low score on a test dataset indicates that the test data set
is not representative of the training dataset. In this case,
consider adding rows to the training dataset that better
represent the test dataset.
HeatWave ML supports a variety of scoring metrics to help you
understand how your model performs across a series of
parameter descriptions and supported metrics, see
Section 3.8.6, “ML_SCORE”.
ensure that the model you want to use is loaded; for example:
CALL sys.ML_MODEL_LOAD(@census_model, NULL);
For information about loading models, see Section 3.7.2, “Loading Models”.
The following example runs
to compute model quality using the
CALL sys.ML_SCORE('heatwaveml_bench.census_validate', 'revenue', @census_model, 'balanced_accuracy', @score);
heatwaveml_bench.census_validateis the fully qualified name of the validation dataset table (
revenueis the name of the target column containing ground truth values.
@census_modelis the session variable that contains the model handle.
balanced_accuracyis the scoring metric. For other supported scoring metrics, see Section 3.8.6, “ML_SCORE”.
@scoreis the user-defined session variable that stores the computed score. The
ML_SCOREroutine populates the variable. User variables are written as
@. The examples in this guide use
@scoreas the variable name. Any valid name for a user-defined variable is permitted (e.g.,
To retrieve the computed score, query the
@score session variable.
SELECT @score; +--------------------+ | @score | +--------------------+ | 0.8188666105270386 | +--------------------+