WL#7567: Handlerton MDL callback

Implement support for the server to be used for DDL statements in a
configuration where objects(tables, triggers and so forth) are shared between
servers, extending the metadata locking to cover DDL statements in sessions also
on other servers which share the same objects. Since objects shared between
servers in MySQL are implemented in the storage engine layer, the storage engine
plugin(s) must be involved to allow statements in sessions on other servers to 
be
locked or unlocked.

Quoted from https://dev.mysql.com/doc/refman/5.6/en/metadata-locking.html:
"MySQL uses metadata locking to manage access to objects (tables, triggers, and
so forth). Metadata locking is used to ensure data consistency ..."

"To ensure transaction serializability, the server must not permit one session
to perform a data definition language (DDL) statement on a table that is used in
an uncompleted explicitly or implicitly started transaction in another session.
The server achieves this by acquiring metadata locks on tables used within a
transaction and deferring release of those locks until the transaction ends. A
metadata lock on a table prevents changes to the table's structure. This locking
approach has the implication that a table that is being used by a transaction
within one session cannot be used in DDL statements by other sessions until the
transaction ends."

This worklog describes how it should be possible for individual storage engine
plugin(s) to hook into the metadata locking in order to implement support for
ensuring that the metadata of the same object may not be modified in more than
one server at a time.

To limit the scope and impact of this feature each server should maintain it's
own state in regards to metadata locking and only when a lock which allows
changes to metadata(ie. exclusive lock) there is a need to involve the storage
engine plugin(s).


When the server detects that metadata is to be changed by a DDL statement it
need to involve the storage engine plugin(s) allowing them to ensure that it's
not allowed to change the metadata of other servers in the same configuration.
This should be done in such a way that concurrent DML or DDL statements on other
objects are not prevented.

User Documentation
==================

New Performance Schema lock status documented here:
http://dev.mysql.com/doc/refman/5.7/en/metadata-locks-table.html

F-1:   The storage engine plugin (hereafter called handlerton) must be
       notified before metadata lock allowing metadata to be changed,
       i.e. exclusive lock, is taken on an object. The handlerton
       must also be notified about start of ALTER TABLE statement,
       which might be expensive to abort at the stage when it
       acquires exclusive metadata lock.
F-1.1: The handlerton should be notified about such lock requests for
       the following types of objects: TABLE, FUNCTION, PROCEDURE,
       TRIGGER, EVENT, SCHEMA and TABLESPACE.
F-1.2: Information making it possible to determine the type of object
       and it's name should be made available.
F-1.3: The handlerton should be able to decide whether or not locking of
       the requested object (or starting ALTER TABLE on the table)
       is currently allowed or not. When the handlerton decides that
       locking is not allowed the statement should fail with an error,
       similarly to other scenarios in which metadata locking fails
       without handlerton involvement.
       New error code and message to be introduced for this purpose.
F-1.4: The handlerton should be reponsible to interpret any errors
       which occurs while it tries to lock on other servers. Depending
       on error and handlerton settings, it should decide wheter or
       not to allow lock.
F-1.5: It should be possible for the handlerton to push warnings
       describing status of the lock operation.

F-2:   The handlerton must be notified after metadata lock allowing
       metadata to be changed is released. The handlerton also must
       be notified about end of ALTER TABLE statement execution.
F-2.1: Information making it possible to determine the type of object
       and it's name should be made available.
F-2.2: It should be possible for the handlerton to push warnings
       indicating status of the unlock operation.
F-2.3: There is no need to try and handle errors from the handlerton
       unlock operation. Should an error occur, processing should
       continue and the next handlerton(if any) should be unlocked.

F-3:   While taking or releasing handlerton lock the server should
       not prevent concurrent DML or DDL statements on other objects.
       This is to allow the handlerton to perform any possible action
       (like waiting on locks, disk reads or network communication)
       while handling the lock/unlock request. 

F-4:   The handlerton does not need to be involved if the object to be
       changed is temporary(since such objects are only visible in one
       server).

F-5:   All the metadata locks where the handlerton is involved should
       be visible and identifiable in P_S.METADATA_LOCKS table if
       instrumentation for metadata locks is enabled. This will make
       it possible to verify the functionality as well as diagnose
       potential problems.

F-6:   In order to avoid potential locking conflicts the order of
       involving the handlerton and performing the metadata lock in
       the server must be consistent and the handlerton need to
       always be involved first when taking lock and last when releasing
       lock.

Rationale for additional notification in case of ALTER TABLE
------------------------------------------------------------
Since in case of ALTER TABLE statement upgrade to exclusive metadata
lock happens late in the process of statement execution it is going
to be expensive to abort statement execution as result of failed
SE notification at this point. To alleviate this issue we are going
to additionally notify handlerton at the start of ALTER TABLE
on base table and at its end. This will give SE chance to abort
execution of ALTER TABLE early in the process without wasting
precious resources.


Changes to SE API
-----------------

Two new function pointers are to be added to handlerton struct.
Engines which are interested in notifications about acquire/release
of exclusive metadata lock and about start/end of ALTER TABLE
need to initialize these members to point to notification functions.
Also enum allowing to differentiate pre-event notification calls
from post-event notification calls will be added.

enum ha_notification_type { HA_NOTIFY_PRE_EVENT, HA_NOTIFY_POST_EVENT };

struct handlerton {
  
  ...

  /**
    Notify/get permission from storage engine before acquisition or after
    release of exclusive metadata lock on object represented by key.

    @param thd                Thread context.
    @param mdl_key            MDL key identifying object on which exclusive
                              lock is to be acquired/was released.
    @param notification_type  Indicates whether this is pre-acquire or
                              post-release notification.

    @note Notification is done only for objects from TABLESPACE, SCHEMA,
          TABLE, FUNCTION, PROCEDURE, TRIGGER and EVENT namespaces.

    @note Problems during notification are to be reported as warnings, MDL
          subsystem will report generic error if pre-acquire notification
          fails/SE refuses lock acquisition.
    @note Return value is ignored/error is not reported in case of
          post-release notification.

    @note In some cases post-release notification might happen when
          there were no prior pre-acquire notification. For example,
          when SE was loaded after exclusive lock acquisition, or when
          we need notify SEs which permitted lock acquisition that it
          didn't happen because one of SEs didn't allow it (in such case
          we will do post-release notification for all SEs for simplicity).

    @return False - if notification was successful/lock can be acquired,
            True - if it has failed/lock should not be acquired.
  */
  bool (*notify_exclusive_mdl)(THD *thd, const MDL_key *mdl_key,
                               ha_notification_type notification_type);


  /**
    Notify/get permission from storage engine before or after execution of
    ALTER TABLE operation on the table identified by the MDL key.

    @param thd                Thread context.
    @param mdl_key            MDL key identifying table which is going to be
                              or was ALTERed.
    @param notification_type  Indicates whether this is pre-ALTER TABLE or
                              post-ALTER TABLE notification.

    @note This hook is necessary because for ALTER TABLE upgrade to X
          metadata lock happens fairly late during the execution process,
          so it can be expensive to abort ALTER TABLE operation at this
          stage by returning failure from notify_exclusive_mdl() hook.

    @note This hook follows the same error reporting convention as
          @see notify_exclusive_mdl().

    @note Similarly to notify_exclusive_mdl() in some cases post-ALTER
          notification might happen even if there were no prior pre-ALTER
          notification.

    @note Post-ALTER notification can happen before post-release notification
          for exclusive metadata lock acquired by this ALTER TABLE.

    @return False - if notification was successful/ALTER TABLE can be proceed.
            True - if it has failed/ALTER TABLE should be aborted.
  */
  bool (*notify_alter_table)(THD *thd, const MDL_key *mdl_key,
                             ha_notification_type notification_type);

  ...

};

We will also add the following two helper functions to handler.h:

bool ha_notify_exclusive_mdl(THD *thd, const MDL_key *mdl_key,
                             ha_notification_type notification_type);
bool ha_notify_alter_table(THD *thd, const MDL_key *mdl_key,
                           ha_notification_type notification_type);

These functions are to be used by SQL-layer to notify ALL interested
(i.e. implementing the above calls) SEs about respective events.


Changes to MDL subsystem and SQL-layer
--------------------------------------

Code in MDL subsystem is to be extended to call ha_notify_exclusive_mdl(...,
HA_NOTIFY_PRE_EVENT) each time we are about to acquire or upgrade to X metadata
lock. If one of SEs refuses lock acquisition MDL subsystem will report
ER_LOCK_REFUSED_BY_ENGINE error.

We will call ha_notify_exclusive_mdl(..., HA_NOTIFY_POST_EVENT) after releasing
or downgrading from X lock. We will also do post-release notification
in cases when we have successfully done pre-acquire notification but
then failed to acquire X metadata lock for some reason. Errors during
post-release notification are ignored.

Similar changes are to be done to ALTER TABLE implementation.


Error message
-------------

New ER_LOCK_REFUSED_BY_ENGINE error and corresponding error message is added.


P_S integration
---------------

It is responsibility of SE to change connection status and PSI stage if
its implementation of notification hook is going to wait for some resource
or perform some long operation. MDL subsystem or ALTER TABLE implementation
won't do anything about this (as they don't know if notification will
involve waiting or not).

Pre-acquire notification and post-release phases in metadata lock
acquisition/release cycle will be reflected by new values in
P_S.METADATA_LOCKS.STATUS column. We will use "PRE_ACQUIRE_NOTIFY"
and "POST_RELEASE_NOTIFY" statuses for pre-acquire and post-release
phases correspondingly. It is responsibility of MDL subsystem to
notify P_S instrumentation about entering/leaving these phases.


Performance considerations
--------------------------

Since notification on exclusive metadata lock acquisition (and ALTER
TABLE notification) will be done only during DDL execution decrease in
performance/scalability caused by such notification (e.g. due to fact
that we acquire LOCK_plugin for it) should not be a concern.