WL#6432: MySQL Fabric: Provisioning using Nova

GOAL
====
The goal of this WL is to exploit the Nova Interface, which is responsible for 
managing compute instances, to easily expand and shrink the number of servers in a 
group when MySQL Fabric is deployed within an OpenStack Cloud Provider.

The Nova Service along with the Glance Service stores "golden images", which might 
be used to start computation instances in the cloud. The image is essentially a 
bootable operating system that can be customized through an automation script 
known as cloud-init. The script runs while the instance boots for the first time 
and might be responsible for installing a MySQL Server, for example, if it is not 
already present in the image.

This cloud-init script is portable among different Linux flavors and there is a 
similar solution for Windows as well. It also might be easily integrated with 
other automation tools such as Puppet, Juju, etc.

REFERENCES
==========

- Cloud-init:
    https://help.ubuntu.com/community/CloudInit

- Python bindings to the OpenStack Nova API:
    http://docs.openstack.org/developer/python-novaclient/

- OpenStack Compute Administration Guide
    http://docs.openstack.org/grizzly/openstack-compute/admin/content/index.html


RELATED BUGS
============

. BUG#18353424 : SUPPORT MULTIPLE COMMAND-LINE OPTIONS (Fixed)

PROPOSAL/SOLUTION
=================
The current solution will be built upon the Nova Python Interface. However to 
avoid cluttering the MySQL Fabric Interface, we will not expose all methods, 
functions and parameters. Only three groups of commands will be introduced: 
provider, machine and snapshot.

A provider identifies a cloud operator and technology in use, an access point 
and defines credentials to access it:

  . Name - The provider's name must be unique and contain information on the 
    technology used by the provider. For example, openstack.rackspace.xxx.

  . Access Point - Address to access the provider, e.g http://address:5000/v2.0.
  
  . User/Passowrd - Credentials to have access to the system.
 
  . Tenant - On behalf of who the requests will be executed.

There will be a library for each technology, c. i.e. OpenStack, Amazon, etc. In 
this WL, only the OpenStack will be supported though. In order to add support 
to 
a new type of provider, developers must create a module within the providers 
package and implement the appropriate interfaces. See details further down.

A machine is a virtual machine which runs an operating system and additional 
software such as a MySQL Server and is installed from an image. To spawn a new 
virtual machine, users have to provide some key information:

  . Image - The image that will be used to create the new virtual machine's 
    environment.

  . Flavor - Virtual machine template which provides information on disk, cpu, 
    memory.

  . Security - Identifies who is going to access the machine and how.

  . Network - Sets the network devices that will be attached to the machine and
    associated IP addresses.

  . Device - Sets the disk devices (e.g. swap) that will be attached to the 
    machine.

  . Customization - Set scripts that will be executed while booting a machine
    for the first time and files that will be replaced or customized.

  . Metadata - Information that identifies the machine created.

It shall be possible to clone a running machine so that we can use the snapshot 
to spawn new virtual machines. While creating a virtual machine, users will 
specify some image's properties and flavor's properties to identify the image 
and flavor that will be used. It is possible to create a new image (i.e. a 
snapshot) upon a machine.

Images have several properties but only the following will be used:

  . name - Free form human readable name given to the image.

  . id - Automatically generate UUID of the image.

  . minDisk - Minimum virtual root disk size in gigabytes required by the
              image.

  . minRam - Minimum virtual machine memory in megabytes required by the
             image.

Flavors have several properties but only the following will be used:

  . name - Free form human readable name given to the image.

  . id - Automatically generate UUID of the image.

  . vcpus - Number of virtual CPUs to use.

  . ram - Virtual machine memory in megabytes.

  . disk - Virtual root disk size in gigabytes.

  . swap - Amount of swap space in megabytes to use.

NOTES/OPEN ISSUES
=================

1 - The current solution will not support authentication using tokens.

    . http://docs.openstack.org/api/quick-start/content/

2 - The current solution will not support SSL between MySQL Fabric and a 
    provider.

    . http://docs.openstack.org/security-guide/content/ch024_authentication.html

4 - We will not change the "server clone", "sharding move_shard" and "sharding 
    split_shard" because they deal with a MySQL Server and the changes 
    introduced by this WL deal with Virtual Machines.

    In the future, we will introduce other ways of cloning a MySQL Server, i.e. 
    doing backups and restores, such as MySQL Enterprise Backup, etc and then 
    eventually the aforementioned commands will be changed.

5 - We need to design a better infra-structure to search for resources in the 
    different sub-systems and update properties.

6 - The parameters files block_device, private_nics and public_nics are not 
    supported yet, although they are mentioned and described in this WL. We are 
    currently investigating whether we are going to support them or not.

MYSQL FABRIC COMMANDS
=====================

mysqlfabric provider register
-----------------------------
. Syntax:

  provider register     
  --provider_type=
  --default_image=
  --default_flavor=
  --synchronous=

. Positional arguments and options:

  . provider_id - Provider's identification.

  . username - User to access the provider.

  . password - Password to access the provider.

  . url - Provider's access point.

  . tenant - Provider's tenant who owns resources.

  . provider_type - Type of the provider such as OPENSTACK which is the default.

  . default_image - Default image to boot a machine.
  
  . default_flavor - Default virtual machine to boot a machine.

  . synchronous - Whether requests should be asynchronously executed.

. Behavior:
  
  If the provider already exists the "ProviderError" is raised.

. Example(s):

  provider register openstack.rackspace.xxx user password 
  http://localhost:5000/v2.0 user

  provider register openstack.rackspace.xxx user password 
  http://localhost:5000/v2.0 user --default_image "Ubuntu 14 MySQL 5.6"
  --default_flavor medium_machine 

mysqlfabric provider unregister
-------------------------------
. Syntax:

  mysqlfabric provider unregister 
  --synchronous=

. Positional arguments and options:

  . provider_id - Provider's identification.

  . synchronous - Whether requests should be asynchronously executed.

. Behavior

  If the provider does not exist or is associated to a machine the
  ProviderError is raised.

. Example(s):

  mysqlfabric provider unregister openstack.rackspace.xxx


mysqlfabric provider list
-------------------------
. Syntax:
 
  mysqlfabric provider list
  --provider_id=

. Positional arguments and options:

  . provider_id - Provider's identification.

. Behavior:

  This command returns a list with information on all providers or a specific
  provider which is identified by the provider_id option.

. Example(s):

  mysqlfabric provider list

  mysqlfabric provider list --provider_id openstack.rackspace.xxx


mysqlfabric machine create
---------------------------
. Syntax:

  mysqlfabric machine create 
  
  --image=
  --flavor=
  --number_machines= 
  --availability_zone=
  --key_name=
  --security_groups=
  --private_network=
  --public_network=
  --userdata=
  --swap=
  --block_device=
  --scheduler_hints=
  --private_nics=
  --public_nics=
  --meta=
  --files=
  --skip_store
  --wait_spawning
  --synchronous

. Positional arguments and options:

  . provider_id - Provider's identification.

  . image - Image to boot the mahine.

    This is a multivalue option that defines properties of the image we are 
    searching for. Example: --image mindisk=xx --image minram=yy or --image 
    name=zzz.

  . flavor - Virtual machine to boot the instance.

    This is a multivalue option that defines properties of the image we are 
    searching for. Example: --flavor vcpus=xx --flavor ram=yy or --flavor
    name=zzz.

  . number_machines - Number of machines that will be created.

  . availability_zone - Name of the availability zone where the machine will
    be installed.

  . key_name - Name of a public key previously stored in the cloud which will 
    be used to access the compute instance.

    If we need to access the running machine through SSH, this is a better
    solution than using a plain password.

  . security_groups - List of security groups that will have access to the
    compute instance separated by comma.
   
  . private_network - Name of the private network where the compute instance
    will be placed in.

    If there is more than one private network, we need to specify where the 
    private address will be fetched from. This parameter assumes that the
    IP will be provided through a DHCP Server and conflicts with the "nics"
    parameter.

  . public_network - Name of the public network that will have access to the
    compute instance.

    If we need to assign a public address to a compute instance, we need to
    specify where the floating address will be fetched from.

  . userdata - data file that will be used by the cloud-init solution.

    The file which will be used by the cloud-init solution to customize the
    image and automate its deployment.

  . swap - Size in MB of the swap block device that will be created and
    attached to the machine. 

  . block-device - Block device mapping.

    The parameter has the following format: =::
    :

  . scheduler_hints - Arbitrary key/value pairs to the scheduler for custom 
    use.

    This is a multivalue option. Example: --scheduler_hints key=value
    --scheduler_hints key=value.

  . private_nics - Create network interface cards on the server. 

    The parameter has the following format: :

        . net-id - NIC to network with this UUID (required if no port-id),

        . v4-fixed-ip - IPv4 fixed address for NIC (optional),

        . port-id: NIC to port with this UUID (required if no net-id)

  . public_nics - Create network interface cards on the server.

    The parameter has the following format: :

        . net-id - NIC to network with this UUID (required if no port-id),

        . port-id: NIC to port with this UUID (required if no net-id)

  . meta - Record arbitrary key/value metadata.

    This is a multivalue option. Example: --meta key=value --meta key=value.

    This is a multivalue option. Example: --file dst-path=src-path --file
    dst-path=src-path.

  . skip_store: Do not store information on machine(s) into the state store. 
    Default is False.

  . wait_spawning - Whether the execution should wait until a machine is
    fully operational.

  . synchronous - Whether requests should be asynchronously executed.

. Behavior:

  This command builds upon the machine abstraction and creates a new compute
  instance. 

  The provider_id is a positional argument which identifies an entry in the 
  "providers" table. It is used to fetch information that will be used to 
  access the cloud provider such Amazon, RackSpace, etc. If the provider does
  not exist,the ProviderError is raised.

  If the flavor is not provided, the default flavor is used. If the image is
  not provided, either the default image is used. By default, only one machine 
  is created but this behavior might be changed through the "number_machines"   
  option. 

  The private_network and private_nics are mutually exclusive. The same happens
  to the public_network and public_nics. 

  The command returns the machines' uuids, IP addresses assigned to the nics 
  and availability zone. If there is a failure, none of the machines are 
  created and the command guarantees that system is left in a clean state and 
  the MachineError is raised.

  Note that although Fabric already provides the ability to process a request
  asynchronously, it possible to specify whether it wait until a machine is
  fully operational or not.

  We are building upon the current Nova Interface which does not support
  some features:

   . The search operation will support only the equal operator.

   . It will not be possible to search for flavor's custom properties.

   . It will not be possible to search for image's custom properties.

   . Images and flavors have several properties but only a limited set will
     be supported. Please, see previous section for a list.

. Examples:

  mysqlfabric machine create openstack.rackspace.xxx

  mysqlfabric machine create openstack.rackspace.xxx --image name=image
  --flavor name=flavor --security_groups "default, high-security"

mysqlfabric machine destroy
---------------------------
. Syntax:

  mysqlfabric machine destroy  
  --skip_store
  --force
  --synchronous

. Positional arguments and options:

  . provider_id - Provider's identification.

  . machine_uuid - Machine's UUID.

  . skip_store - Proceed anyway if there is no information on the machine in
    the state store. Default is False.

  . force - Remove a reference to the machine from the state store and ignore 
    possible errors.

  . synchronous - Whether requests should be asynchronously executed.

. Behavior:

  This command destroys a compute instance. If for some reason, the compute 
  instance is not found in the provider and we want to remove any reference to 
  it from MySQL Fabric, the "force" option must be used. Automatically, all
  associated snapshots are destroyed and public IP addresses released.

. Example(s):

  mysqlfabric machine destroy openstack.rackspace.xxx
  9f3d9729-f220-11e3-adbb-b86b23dc9477

  mysqlfabric machine destroy openstack.rackspace.xxx
  9f3d9729-f220-11e3-adbb-b86b23dc9477 --force


mysqlfabric machine list
------------------------
. Syntax:

  mysqlfabric machine list 
  --skip_store

. Positional arguments and options:

  . provider_id - Provider's identification.

  . skip_store - Proceed anyway if there is no information on the machine in
    the state store. Default is False.

. Behavior:

  This command returns a list with information on all machines associated to
  a provider identified by the provider_id.

. Example(s):

  mysqlfabric machine list openstack.rackspace.xxx


mysqlfabric snapshot create
---------------------------
. Syntax:

  mysqlfabric snapshot create  
  --skip_store
  --wait_spawning
  --synchronous

. Positional arguments and options:

  . provider_id - Provider's identification.

  . machine_uuid - Machine's UUID.

  . skip_store - Proceed anyway if there is no information on the machine in
    the state store. Default is False.

  . wait_spawning - Whether the execution should wait until the snapshot is
    created.

  . synchronous - Whether requests should be asynchronously executed.

. Behavior:

  Create a snapshot, which might be used to create new and fresh machines, upon 
  a machine.

. Example(s):

  mysqlfabric snapshot create openstack.rackspace.xxx
  9f3d9729-f220-11e3-adbb-b86b23dc9477 


mysqlfabric snapshot destroy
----------------------------
. Syntax:

  mysqlfabric snapshot destroy  
  --skip_store
  --synchronous

. Positional arguments and options:

  . provider_id - Provider's identification.

  . machine_uuid - Machine's UUID.

  . skip_store - Proceed anyway if there is no information on the machine in
    the state store. Default is False.

  . synchronous - Whether requests should be asynchronously executed.

. Behavior:

  Destroy all snapshots created upon a running machine.

. Example(s):

  mysqlfabric snapshot destroy openstack.rackspace.xxx
  9f3d9729-f220-11e3-adbb-b86b23dc9477 


ERRORS
======

. MachineError - Error processing a request that requires access to provider's 
  machine or fetching and updating information on it.

. ProviderError - For example, provider does not exist in the configuration.


TABLES
======

. TABLE providers:

  . provider_id VARCHAR(256) NOT NULL
  
  . type INT NOT NULL
  
  . username VARCHAR(100) NOT NULL
  
  . password VARCHAR(128) NOT NULL
  
  . url VARCHAR(256) NOT NULL
  
  . tenant VARCHAR(100) NOT NULL

  . default_image VARCHAR(256)

  . default_flavor VARCHAR(256)


. TABLE machines

  . machine_uuid VARCHAR(40) NOT NULL

  . provider_id VARCHAR(256) NOT NULL

  . av_zone VARCHAR(256)

  . addresses TEXT


FILES
=====

The following files will be introduced:

  . lib/mysql/fabric/machine.py - Contains the Machine class which is used to
    access the state store.

  . lib/mysql/fabric/provider.py - Contains the Provider class which is used to 
    access the state store.

  . lib/mysql/fabric/providers/__init__.py - Contains classe(s) that must be
    inherited by the supported providers and code to check whether required
    libraries are installed or not.
 
  . lib/mysql/fabric/providers/null.py - Code that emulates a provider and is
    use to test the system.

  . lib/mysql/fabric/providers/openstack.py - Code to access an OpenStack 
    provider.

  . lib/mysql/fabric/services/machine.py - Commands to manage machine, e.g.
    creating and destroying machine. 
  
  . lib/mysql/fabric/services/provider.py - Command to manage providers, e.g.
    registering and unregistering providers.


ADDING SUPPORT TO NEW PROVIDERS
===============================

There will be a library for each technology, i.e. OpenStack, Amazon, etc. In 
this WL, only the OpenStack will support will be introduced though.

To add support to a different cloud provider, we need to drop a module within 
the providers package and define a *configure_provider* function within the 
module. This function will be responsible for checking whether the necessary 
packages to access the provider are installed and will return the following 
information:

- Unique Provider's Identification which is a string.
- Reference to a concrete calls built upon the AbstractMachineManager.
- Unique Provider's index which is an integer.

For example, the openstack.py module has the following *configure_provider* 
function::

  def configure_provider():
    import novaclient.client
    return ("OPENSTACK", MachineManager, 2)

Each supported provider must return a reference to a concrete class that 
inherits from the AbstractMachineManager class defined in this module. This 
class is an entry point to a set of methods to access a provider.