WL#9682: InnoDB: Support cloning encrypted and compressed database

Affects: Server-8.0   —   Status: Complete

Support clone of Encrypted tables, general tablespaces, undo tablespaces and
redo logs. It should work with local key management service and with centralized
key management solution, i.e. “MySQL Enterprise Transparent Data Encryption (TDE)”.

Essentially the tablespace keys from donor would be transparently re-encrypted
with master key at recipient. SSL connection would be enforced to ensure safe
transfer of database keys over network.

For compression, page data would be compressed and holes would be punched at
recipient when the target files system supports punching holes.
Functional Requirements:
F-1: CLONE SQL must ensure that encrypted tables and general tablespaces are
cloned as encrypted and is operational after clone.

*For "CLONE LOCAL" and locally managed key ring the same key ring needs to
be used for starting cloned server. For "Centralized key management solution" it
is not an issue because same key ring is accessed by all.

F-2: CLONE SQL must ensure that page compression enabled tables (Transparent
Page Compression) are cloned and operational after clone.

F-3: CLONE SQL must clone encrypted redo log.

F-4: CLONE SQL must clone encrypted UNDO tablespace.

F-5: Must throw error if redo log or undo log encryption is set while clone is
in progress. 
    SET GLOBAL innodb_redo_log_encrypt = ON;
    SET GLOBAL innodb_undo_log_encrypt = ON;

Non-functional requirements:
NF-1: Security: Must ensure SSL connection while transferring tablespace
encryption keys over network.

NF-2: Space: Shall ensure that page compressed tables have pages compressed and
hole punched in cloned database if the target file system supports punching hole.

*If the target file system doesn't support hole punching, the page compressed
tables would occupy same size as the file length on disk in cloned database.
Encryption Design:
1. During FILE_COPY, all file data is cloned as it is.
2. All encrypted tablespace header page is added for "PAGE_COPY"
3. Innodb informs clone plugin to ensure SSL.
4. During PAGE_COPY, at donor, tablespace encryption key is decrypted using
master key before sending the header page.
5. During PAGE_COPY, at recipient, tablespace encryption key is encrypted with
donor master key.
6. Same way encryption key is decrypted and encrypted for redo log.

7. Concurrency with change in encryption state.
a. DDLs are blocked during clone and user tablespace state don't change from
encrypted to unencrypted.

b. For redo and undo tablespace, while changing the encryption configurations,
we check and throw error if clone is in progress. If encryption is in progress,
clone waits for it to complete before starting.

c. Key rotation: Clone decrypts the key in header using master key before
transferring data. Should not have any impact for key rotation.

Compression Design:
During FILE_COPY and PAGE_COPY, at recipient, Innodb asks for the data buffer
from clone plugin and attempts to punch holes based on compressed data size in
each page.

Interface changes:
[I-1: Error Codes]:

  "Clone Donor plugin %.128s is not active in Recipient."

Thrown if key ring plugin is not installed in recipient. 

  "Clone needs SSL connection for encrypted table."

Thrown if SSL is not available or user used "NO SSL" option. 

  "Clone Configuration <"FS Block Size">: Donor value:  is different from
Recipient value: ."

Thrown if there are compressed or encrypted tables and file system block sizes
don't match.

  "Concurrent clone in progress. Please try after clone is complete."

Thrown if trying to set encryption for redo and undo log while clone is in progress.
  SET GLOBAL innodb_redo_log_encrypt = ON
  SET GLOBAL innodb_undo_log_encrypt = ON

[I2: Internal: handlerton security]  

class Ha_clone_cbk {
  /** Mark that data needs secure transfer. */
  void set_secure() { m_flag |= HA_CLONE_SECURE; }

  /** Check if data needs secure transfer. */
  bool is_secure() const { return (m_flag & HA_CLONE_SECURE); }

  /** Data needs to be transferred securely over SSL connection. */
  const int HA_CLONE_SECURE = 0x08;

a. SE/Innodb uses set_secure() on the callback object to indicate it needs
secure data transfer when encrypted data is involved.

b. Clone Plugin at donor would check and throw error if recipient established
non SSL connection.

[I3: Internal: handlerton callback]

/** Callback to get data in buffer.
@param[out]  to_buffer  data buffer
@param[out]  len        data length
@return error code */
virtual int apply_buffer_cbk(uchar *&to_buffer, uint &len) = 0;

Added buffer callback for clone apply. Usually clone would use file callback
during apply where SE provides the file descriptor to clone to write data
directly. For encryption header page Innodb needs to modify the page by
encrypting the key before write. For compressed page Innodb attempts to punch
hole after write.
Here is a reference to data structure and functions involved in the key flow.

1. Clone file metadata information to send across for compression and encryption.
struct Clone_File_Meta {
  /** File compression type */
  Compression::Type m_compress_type;

  /* File encryption type */
  Encryption::Type m_encrypt_type;

  /** File system block size. */
  uint32_t m_fsblk_size;

2. Decrypt tablespace and log key from master key at donor.

3. Re-encrypt with master key at recipient.

4. Compressed page handling at recipient 

5. Concurrency with dynamic enabling of redo and undo encryption.
  /** Mark to indicate that new clone operations should wait.
  @return true, if no active clone and mark is set successfully */
  bool mark_wait();

  /** Free the wait marker. */
  void mark_free();

  /** Wait for marker to get freed.
  @param[in,out]       thd     user session
  @return, error if timeout */
  int wait_for_free(THD *thd);