WL#9289: InnoDB: Support Transparent Data Encryption for Undo Tablespaces

Affects: Server-8.0   —   Status: Complete

This work will provide encryption support for undo tablespaces.

Basically, for encryption, undo log tablespace has no difference with regular 
tablespace. The special thing is how to en/disable undo log encryption.

1:We added a new global variable innodb_undo_log_encrypt=ON/OFF for en/disable 
undo log encryption.

And after user enable undo log encryption, the undo log pages will be encrypted 
when they're writing into disk. the previous undo log pages which is already on
disk will be left as not encrypted status.
On the other hand, after it's disable, the old undo log pages will be leave as
encrypted status, and the new undo log pages will not be encrypted.

If innodb_undo_log_encrypt is enabled in bootstrap, we use a default master key
for encrypting undo log tablespace key and iv. It's because there's no necessary
information like server uuid in bootstrap. And in the next master key rotation, 
the default master key will be thrown away.

2:In this worklog, we only support encryption of independent undo tablespaces.
Since currently, we don't support encryption of shared tablespaces, so,
we will not support encrypting undo log pages in system tablespace.
This feature will be provided in the coming worklog.

3:Same as regular tablespace, the encryption metadata will be stored in the first
page of data file.

And in the future work undo tablespace metadata is stored in the Global Data
Dictionary dd::Tablespace object, and loaded on server startup. Then, the
encryption metadata can be stored in dd::Tablespace::options or
dd::Tablespace::se_private_data. The main thing that is missing is the SQL
interface to specify encryption when 
creating a tablespace.

4:Performance impact of undo log encryption will be less than 10%.

Requirements

F1 - Support AES encryption
F2 - En/disable undo tablespace encryption by set innodb_undo_log_encrypt=ON/OFF.
F3 - Support Key rotation.
F4 - Read encryption information from header of undo* properly (recovery 
after crash)
F5 - Don't support encryption of undo log in system tablespace.

Non-Functional requirements
NF-1 - Performance impact of undo log encryption will be less than 10%
1:En/disable undo log tablespace encryption:
Introduce a global system variable innodb_undo_log_encrypt
  ie:set global innodb_undo_log_encrypt=ON, start encrypt undo log pages.

2:En/decrypt undo log pages:
Since the undo log pages has not different format with normal data page,
we don't need to do anything special for this, just use the exist en/decrypt 
functions in class Encryption. But we need to set the proper encryption
metadata into IORequest.

3:Store encryption metadata:
No difference with regular tablespaces.
The encryption metadata will be stored in the header of data file, which is in
the page 0, and The encryption metadata information of undo tablespace has the 
same format with it in regular tablespaces. like:
encryption version + master key id + server uuid + encrypted tablespace key + IV


4:Key rotation and recovery
If the undo log tablespace has been set as encrypted, we will rotate the undo log 
tablespace key, even innodb_undo_log_encrpyt is OFF. Since it is possible that 
some of undo log pages are still in encrypted status, and they need encryption
metadata to decrypt.
1:En/disable undo log encryption:
Add new global variable innodb_undo_log_encrypt definition in ha_innobase.cc:
 
static MYSQL_SYSVAR_BOOL(undo_log_encrypt, srv_undo_log_encrypt,
  PLUGIN_VAR_OPCMDARG,
  "Enable or Disable Encrypt of UNDO tablespace.",
  NULL, NULL, FALSE);

2:Add a function to enable undo log encryption in server startup.
/** Try to enable encryption of an undo log tablespace.
@param[in]     space           undo log tablespace
@return DB_SUCCESS if success */
static
dberr_t
srv_undo_tablespace_enable_encryption(
       fil_space_t*    space);