These release notes were created with the assistance of HeatWave GenAI.
HeatWave AutoML now supports more efficient memory usage, reducing the footprint of its processes during training and explanations for large datasets. (WL #16606)
HeatWave GenAI supports enhanced unstructured document ingestion with Optical Character Recognition (OCR) now enabled by default. (WL #16584)
-
The
ML_RAG
andML_RAG_TABLE
routines now supports advanced retrieval options, enabling more precise and context-aware results. Three new parameters have been added to the routines:max_distance
for filtering segments based on distance from the input query.percentage_distance
for adaptive distance filtering of segments from the input query.segment_overlap
for retrieving segments adjacent to the segment nearest to the input query for providing continuous context.
These new parameters are integrated into the existing
@options
parameter, and can be enabled through a new JSON object parameter,@retrieval_options
. (WL #16543)
-
HeatWave Lakehouse now supports selective load, which lets you update the file paths comprising a Lakehouse table by manipulating Lakehouse metadata. The changes are incrementally applied to the table data. This feature enables you to add or remove Object Storage bucket data directories from a Lakehouse table without reloading the entire table. Changes are executed transactionally, ensuring data consistency and query isolation.
For more information, see Adding or Removing Files Using Selective Load. (WL #16531)
-
HeatWave Lakehouse now supports enhanced error and warning messages, providing more data points for easier debugging. This update adds row numbers and character offsets to relevant warning messages in non-strict mode during secondary load, as well as updates byte offset text messages to point to column positions. JSON datatype error and warning messages have also been improved with error and warning offset information within the column.
For more information, see HeatWave Lakehouse Error Messages. (WL #16530)
-
HeatWave Lakehouse now supports efficient loading of Parquet files with large row groups (larger than 500MB and upto 10GB) using large shapes. This helps in overcoming previous memory budget constraints and significantly reducing load times. This enhancement also introduces row group sharing, caching, and synchronization mechanisms to minimize network requests, decoding time, and memory usage during transformation.
For more information, see Lakehouse Limitations for the Parquet File Format. (WL #16512)