WL#6829: Group Replication: Generic Interface for accessing communication infrastructure

EXECUTIVE SUMMARY
=================
this is a simple interface that one can bind to any group 
communication system. This provides flexibility, and helps 
while transitioning from a prototype into a product and 
enables unit testing.

EXTENDED SUMMARY
================

The purpose of this deliverable is to provide the system with an 
abstraction to access multiple group communication toolkits, in 
a generic fashion. Therefore, with the API inplace, one can isolate 
the components logics and bind the communication API to any toolkit 
without affecting the rest of the code. This is similar to how 
Oracle’s Shoal was architected with its “Group Communication Provider 
API” [1].

PROBLEM STATEMENT
=================

We can develop a group communication system (GCS) or use
a 3rd party one. As a prototype we will use a third party GCS. 
We want to be able to easily replace the GCS used without 
affecting other modules.
This worklog aims at formulation of basic GCS services and specifying them
as an abstract class.

REFERENCES
==========
[1] http://shoal.java.net/

*NOTE*, there're no non-functional requirements.

F-1: The set of interfaces shall be compatible with any GCS implementation.

*NOTE*, there's no user visible interface specification.
Interfaces of this WL represent "internal" services that mysql server
instances make use of to provide an ultimate mysql synchrounus
replication or the cluster service to the user.

On a hight level MySQL server interaction with GCS as an application is
specified as the following:


                     +-----------------------+
                     |                       |
                     |  mysqld (application) |
                     |                       |
                     +-+------+-----o-----o--+
                Join _ |      |     |     |
                Leave  | send-|     |-recv|-View_chng
                       |      |     |     |
                  +----o------o-----+-----+-------+
                  |        G     C    S           |
                  +-------------------------------+

`Join' and `Leave' are initiated by the server, `View_chng' is done
by the GCS.
While `Join', `Leave' and `send' appear straighforward functionalities
so the server just calls corresponding functions, `recv' and `View_chng'
can't be invoked so freely. Normally those are called from a dedicated
reciever thread who further notifies interesting parties about events arrived.
So we use a model of callbacks (from GCS pov) in that the server provides
functions to react on messages or GCS membership changes.

Think of the following undefined class-clauses as sort of forward declarations.

class Member;  // (...)
class Message; // (...)
class MemberSet; //  (group_name, std::set)

enum enum_gcs_msg_agreed { GCS_BEST_EFFORT, GCS_RELIABLE, GCS_UNIFORM};
enum enum_gcs_msg_ordered { GCS_UNORDERED, GCS_TOTAL_ORDER };
enum enum_gcs_member_role { GCS_ACTIVE, GCS_OBSERVER };

typedef struct handlers
{
  /**
   * View change handler. It's supposed to get control once
   * the group experiences any change.
   *
   * @param p        Handle of the protocol session to carry out callbacks
   * @param gname    The group name string
   * @param ptr_totl pointer to the new member set
   * @param left     reference to a set of nodes that left
   * @param joined   reference to a set of new arrived nodes
   *
   * @note The handler MUST release memory pointed by the arguments.
   * SHALL there be an empty set, then the View pointer
   * SHALL point to NULL.
   *
   */
  void (*view_change)(Protocol *p, string& gname, Member_set* ptr_totl,
                      Member_set& left, Member_set& joined);

  /**
   *  Refined version of view_change to provide the Primary
   *  Component service to cope with network partitioning (split
   *  brain).
   *
   * @param p        Handle of the protocol session to carry out callbacks
   * @param gname    The group name string
   * @param ptr_totl pointer to the new member set
   * @param left     reference to a set of nodes that left
   * @param joined   reference to a set of new arrived nodes
   * @param quorate is false if the member is not part of a quorate partition or
true
   *                 otherwise.
   */
  void (*primcomp_change)(Protocol *p, string& gname, Member_set* ptr_totl,
                          Member_set& left, Member_set& joined, bool quorate);

  /**
   * @param msg  The message to be delivered
   * @param view View in which delivery happens
   */
  void (*message_delivery)(Message* msg, View* ptr_v);
} Event_handlers;

/**
 * We use the group communication `protocol' as an alias of GCS.
 * This class aggregates interfaces belonging to few GCS categories.
 * Those include Data and Control interfaces as well as Session management.
 * From the pure interfaces perspective there's no limit on the number
 * of the class instances nor groups an instance is allowed to join.
 */
class Protocol
{
  /*
    The class is defined as abstract. Instantiation is left to a derived class.
  */
private:

  Protocol(const Protocol&);

protected:

  Protocol() : handlers(NULL) {};
  virtual ~Protocol() {};

public:

  /**
     The session properties include customizable vector of handlers which 
     gets determined through @c init_session.
  */
  Event_handlers *handlers;

  /**
     This member describes the protocol capabilities. A member
     presents it at group joining and may adopt an older value
     that is compatible with the rest of the group and adapt to it.
     The member plays significant role in upgrading and mixed version
     members in the cluster use cases.
  */
  ulonglong version;

  /**
   * The method conducts session initialization to assign a vector
   * of @c Event_handlers to @c this->handlers.
   *
   * @note this member function MUST be called prior to the
   * @c Protocol::join member function.
   *
   * @param handlers Vector of reactions of the protocol event.
   * @return false if the setting succeeded, true otherwise.
   */
  virtual bool open_session(Event_handlers* handlers_arg)= 0;

  /**
   * Undoes initialization performed by @c init_session.
   * @note This member function MUST be called after leaving the group.
   *
   * @return true if the shutdown operation did not succeed,
   *              false otherwise.
   */
  virtual bool close_session()= 0;

  /**
     @return enum value of the selected protocol
  */
  virtual Protocol_type get_type()= 0;

  /**
   * This member function SHALL broadcast a message to the group.
   * The guarantees of the broadcast delivery SHALL be those as specified
   * on the @c Message::qos and @c Message::order fields.
   *
   * Note that this member function passes the Message instance by
   * reference, thence it retains ownership of the object. It also states
   * that it is constant, so no changes SHALL occur in the implementor of
   * this interface.
   *
   * @param Message the message to be broadcast.
   * @return false if the sending succeeded, true otherwise.
   */
  virtual bool broadcast(const Message& msg)= 0;

  /**
   * This member function SHALL join this member to the specified group.
   * If the member is already joined, then this function is ineffective
   * and SHALL return true.
   *
   * When a server joins a group, it can define its role. It can be an
   * observer or an active participant. If the underlaying group communication
   * toolkit does not support OBSERVER roles and join is issued with
   * OBSERVER as role, then this function shall result in an error,
   * and the join operation will fail.
   *
   * If the underlaying group communication system does not support
   * multiple groups, this function shall return an error and the join
   * operation is ineffective.
   *
   * @param group_name The name of the group to join.
   * @param role The type of role that this member will fulfil in the
   *             group. It is either ACTOR or OBSERVER.
   * @return false if the join operation was successful, true otherwise.
   */
  virtual bool join(const string& group_name,
                    enum_member_role role= MEMBER_ACTOR)= 0;

  /**
   * This member function SHALL remove this member from the currently
   * joined group. If no group is joined, then this function is
   * ineffective and SHALL return true.
   *
   * If the member had joined with role as OBSERVER, then the fact that
   * this member leaves the group SHALL not trigger a view change.
   *
   * @return false if the leave operation succeed, false otherwise.
   */
  virtual bool leave()= 0;

  /**
   * This member function SHALL get the current view on the group.
   *
   * If the member has not joined a group yet, this function SHALL
   * return true.
   *

   * TODO/FIXME ** view allocation **

   * The ownership of the instance of View is put on the caller,
   * thence it MUST release the memory used for this  instance, by
   * calling "delete view;".
   *
   * @return pointer to View object when read successfully, otherwise NULL.
   */
  virtual View* get_view(const string& group_name)= 0;
  virtual View* set_view(View * arg)= 0;
  /**
   * @return true if currently connected *nodes* configuration is the
   * quorate partition.
   */
  virtual bool is_quorate()= 0;

  /**
     The method is to help unit-testing.
  */
  virtual void test_me()= 0;

};